Robot control method, robot, and recording medium

ABSTRACT

A robot control system acquires request information requesting a robot to keep within a prescribed distance from a user and to collect surrounding information, acquires, from the robot, location information indicating a location of the robot and confirmation information indicating that the robot is near the user, and transmits, to the robot, a command for changing a setting of the robot from a first specification to a second specification in response to the request in a case where it is determined, on a basis of the map information, the location information, and the confirmation information, that the user of the robot is accompanying the robot outside a home area of the user, the first specification enabling the robot to collect the surrounding information inside the home area, and the second specification enabling the robot to collect the surrounding information outside the home area.

BACKGROUND 1. Technical Field

The present disclosure relates to a technology that provides the skillsof a robot to external parties.

2. Description of the Related Art

Japanese Unexamined Patent Application Publication No. 2007-264950discloses, for the objective of deterring crime against a specificperson, a robot capable of moving alongside the specific person, inwhich the robot uses a display and a speaker to announce that the robotis monitoring the specific person and to emit a warning when an incidentoccurs.

SUMMARY

However, Japanese Unexamined Patent Application Publication No.2007-264950 has room for further improvement.

In one general aspect, the techniques disclosed here feature a methodfor controlling a robot in a robot control system including a serverthat communicates with the robot of a user and stores map informationindicating a home of the user, the method including: acquiring, from anexternal computer, request information indicating a request for therobot to keep within a prescribed distance relative to a location of theuser of the robot and to collect surrounding information about the userwhen the user is outside a prescribed area including the home of theuser; acquiring, from the robot, location information indicating alocation of the robot and confirmation information indicating that therobot is near the user; and transmitting, to the robot, a command forchanging a setting of the robot from a first specification to a secondspecification in response to the request in a case where it isdetermined, on a basis of the map information, the location information,and the confirmation information, that the user of the robot isaccompanying the robot outside the prescribed area expressing the homeof the user, the first specification enabling the robot to collect thesurrounding information about the user inside the prescribed areaincluding the home of the user, and the second specification enablingthe robot to collect the surrounding information about the user outsidethe prescribed area including the home of the user.

It should be noted that general or specific embodiments may beimplemented as a system, a method, an integrated circuit, a computerprogram, a storage medium, or any selective combination thereof.

According to the present disclosure, further improvement is madepossible.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of an overallconfiguration of an information system according to an embodiment of thepresent disclosure;

FIG. 2 is an illustration of the external appearance of a robotaccording to the embodiment;

FIG. 3 is a block diagram illustrating an example of a configuration ofthe information system according to an embodiment of the presentdisclosure;

FIG. 4 is a flowchart illustrating an example of a process when acompany A and a robot cooperate;

FIG. 5 is a table illustrating an example of relationships between typesof access rights to a robot and permission levels;

FIG. 6 is a table illustrating an example of relationships betweensecurity modes and access rights to a robot in each security mode;

FIG. 7 is a sequence diagram illustrating an example of a process by aninformation system in a case where the entity that determines whether ornot a robot and a user are out is a company B server;

FIG. 8 is a diagram illustrating an example of a process by the companyB server and the robot corresponding to FIG. 7 ;

FIG. 9 is a diagram illustrating an example of a scenario in which asecurity mode is switched from “at home” to “walk”;

FIG. 10 is a diagram illustrating an example of an at-home area;

FIG. 11 is a diagram illustrating an example of an at-home area;

FIG. 12 is a diagram illustrating an example of an at-home area;

FIG. 13 is a diagram illustrating an example of an at-home area;

FIG. 14 is a diagram illustrating an example of a notification that arobot issues to a user;

FIG. 15 is a diagram illustrating an example of a notification that arobot issues to a user;

FIG. 16 is a diagram illustrating an example of a notification that arobot issues to a user;

FIG. 17 is a sequence diagram illustrating an example of a process by aninformation system in a case where the entity that determines whether ornot a robot and a user are out is a company A server;

FIG. 18 is a flowchart illustrating an example of a process by thecompany A server, the company B server, and the robot corresponding toFIG. 17 ;

FIG. 19 is a sequence diagram illustrating an example of an informationprocessing system in a case where the entity that determines whether ornot a robot and a user are out is the robot;

FIG. 20 is a flowchart illustrating an example of a process by the robotcorresponding to FIG. 19 ;

FIG. 21 is a flowchart illustrating an example of a process for updatinga necessary security level in a robot;

FIG. 22 is a diagram illustrating an example of a home screen displayedon a smartphone after launching a company A app;

FIG. 23 is a diagram illustrating an example of a settings screen;

FIG. 24 is a diagram illustrating a settings screen in another exampleaccording to the embodiment;

FIG. 25 is a diagram illustrating a first example of a security mode;

FIG. 26 is a diagram illustrating a first example of a security mode;

FIG. 27 is a diagram illustrating a first example of a security mode;

FIG. 28 is a diagram illustrating the behavior of a robot when movingfrom a home position to an emission position;

FIG. 29 is a flowchart illustrating an example of a process by a robotin a first example of a security mode;

FIG. 30 is a diagram illustrating a second example of a security mode;

FIG. 31 is a flowchart illustrating an example of a process by a robotin a second example of a security mode;

FIG. 32 is a diagram illustrating a third example of a security mode;

FIG. 33 is a diagram illustrating a third example of a security mode;

FIG. 34 is a diagram illustrating a third example of a security mode;

FIG. 35 is a flowchart illustrating an example of a process by a robotin a third example of a security mode;

FIG. 36 is a diagram illustrating a situation in which a robot avoids aperson approaching a user;

FIG. 37 is a flowchart illustrating an example of a process by a robotwhen the robot avoids a person approaching a user; and

FIG. 38 is a diagram illustrating an example of a basic function of anescorting robot.

DETAILED DESCRIPTIONS (Underlying Knowledge Forming Basis of the PresentDisclosure)

Our daily lives are becoming more and more digitized. For example, manypeople carry a smartphone, that is, a personal information communicationterminal. Users have come to install and use a variety of applications(hereinafter referred to as apps) on their smartphones, such as apps formanaging the user's health, apps supporting personal finance management,social communication apps supporting real-time communication withacquaintances, and news apps that curate news from around the worldaccording to the user's personal interests.

On the other hand, although progress has been gradual, mobile devicescapable of performing various movements and tasks autonomously(hereinafter referred to as robots) have also been put into practicaluse. There are robots that assemble and adjust part in factories, robotsthat accurately and rapidly sort objects in distribution centers, robotsthat can carry out specific tasks while also considering the surroundingsituation, and the like. Among these robots, there are also robots thatcollaborate with humans and do specific tasks in place of humans.

The present disclosure helps users live healthy, happy, comfortable,safe, secure, enjoyable, and/or clean lives through cooperative actionbetween a smartphone, that is, an information processing device capableof executing a wide variety of information processing, and a robot, thatis, a mobile device capable of performing a wide variety of movementsand tasks involving the manipulation of objects.

As an example of one such technology related to the cooperative actionbetween a smartphone and a robot, a technology in which a robot escortsa user to ensure the safety of the user when the user goes out for awalk or a jog outside the home is being examined.

However, unlike the situation at home, the situation outside the homevaries considerably, and the current functionality of robots has beeninadequate for appropriately ensuring the safety of a user performingactivities such as taking a walk outside the home.

For example, the disclosure in Patent Literature 1 merely indicatesthat, to increase the effect of deterring crime at nighttime or thelike, a robot lights up the area around a user with backlight lightsource or detects an incident involving the user and uses a display andspeaker to announce that the area is being monitored. Therefore, thetechnology disclosed in Patent Literature 1 is inadequate forappropriately ensuring the safety of a user performing activitiesoutside the home.

The present disclosure has been devised to address problems like theabove.

(1) A method according to one aspect of the present disclosure is amethod for controlling a robot in a robot control system including aserver that communicates with a robot of a user and stores mapinformation indicating a home of the user, the method comprising:acquiring, from an external computer, request information indicating arequest for the robot to keep within a prescribed distance relative to alocation of the user of the robot and to collect surrounding informationabout the user when the user is outside a prescribed area including ahome of the user; acquiring, from the robot, location informationindicating a location of the robot and confirmation informationindicating that the robot is near the user; and transmitting, to therobot, a command for changing a setting of the robot from a firstspecification to a second specification in response to the request in acase where it is determined, on a basis of the map information, thelocation information, and the confirmation information, that the user ofthe robot is accompanying the robot outside the prescribed areaexpressing the home of the user, the first specification enabling therobot to collect the surrounding information about the user inside theprescribed area including the home of the user, and the secondspecification enabling the robot to collect the surrounding informationabout the user outside the prescribed area including the home of theuser.

With this arrangement, when the user of the robot is outside aprescribed area expressing the user's home, a security company usesfunctions provided in the robot to monitor the user from a terminal atthe security company.

In this case, a server of the robot acquires, from the externalcomputer, request information indicating a request for the robot to keepwithin a prescribed distance relative to the location of the user of therobot and to monitor the user when the user is outside the prescribedarea expressing the user's home while escorted by the robot. Moreover,in response to the request, the server transmits to the robot a commandfor changing a setting of the robot to a specification enabling therobot to collect surrounding information about the user while the useris outside the prescribed area expressing the user's home.

Accordingly, the robot can be used continuously for long periods of timewithout consuming power unnecessarily due to keeping the robot set to anadvanced specification that utilizes advanced motor skills and sensorsthat are not strictly necessary at home. On the other hand, when outsidethe home, the setting of the robot is changed to the advancedspecification that utilizes advanced motor skills and sensors that canbe used to ensure the user's safety even at night and when outdoors, andthe robot is made to monitor the surrounding situation, including theuser, appropriately.

With this arrangement, when the user of the robot is outside the home ofthe user, the security company can utilize functions provided in therobot to collect information about the user's surroundings and ensurethe user's safety. In addition, by causing the robot to sense the user'ssurroundings over a wide range and/or with high accuracy and reactpromptly if necessary, the user's safety can be better ensured whileoutside the home.

(2) In the above method, the prescribed area including the home mayinclude an area indicating a location and planar extent of the home.

With this arrangement, the location and planar extent of the home can beused to appropriately represent the area expressing the user's home.

(3) In the above method, the prescribed area including the home mayinclude an area within a prescribed distance from the home.

With this arrangement, a prescribed distance from the user's home can beused to appropriately represent the area expressing the home.

(4) In the above method, the prescribed area including the home mayinclude an area which includes the home and which is set by the user.

With this arrangement, the area indicating the user's home can be set bythe user.

(5) In the above method, the prescribed area including the home mayinclude an area which includes the home and which is set in the robotcontrol system.

With this arrangement, the area indicating the user's home can be set bythe robot control system.

(6) In the above method, the confirmation information may indicate thatthe robot has determined that the robot is near the user on a basis of aresult of recognizing the user by image recognition.

With this arrangement, the presence of the user near the robot can bespecified accurately by image recognition.

(7) In the above method, the confirmation information may indicate thatthe robot has determined that the robot is near the user on a basis of aresult of acquiring, from a terminal carried by the user, currentlocation information about the terminal.

With this arrangement, the current location information of the user'sterminal can be used to accurately specify the presence of the user nearthe robot.

(8) In the above method, the confirmation information may include secondlocation information which indicates a location of the user from therobot and which is determined by the robot on a basis of a result ofrecognizing the user.

With this arrangement, a second location information recognized by therobot and indicating the location of the user from the robot can be usedto accurately specify the presence of the user near the robot.

(9) In the above method, the changed setting of the robot may include asetting enabling a mobility skill provided in the robot to move at ahigher speed compared to a case where the robot is inside the home ofthe user.

With this arrangement, the robot outside the home can move fastercompared to inside the home, and therefore the user's safety outside thehome can be ensured appropriately.

(10) In the above method, the changed setting of the robot may include asetting that turns on illumination equipment provided in the robot toilluminate the ground near the feet of the user who is moving outsidethe home.

With this arrangement, the area around the user's feet is illuminated,and the safety of the user moving outside the home can be ensuredappropriately.

(11) In the above method, the command may include an instruction causingthe robot to output to the robot control system, after the setting ofthe robot is changed, an indication that the setting of the robot hasbeen changed.

With this arrangement, the completion of the setting change of the robotinside the robot can be confirmed by the user of the robot from theoutside of the robot.

(12) In the above robot control method, in a case where it is confirmedthat the external computer has access rights for making the robotperform the request indicated by the request information, the commandmay be transmitted to the robot.

This arrangement makes it possible to prevent a setting change of therobot based on a remote instruction from an external source with noaccess rights to the robot.

(13) In the above method, an indication of whether the external computerhas the access rights may be registered in the server, and the externalcomputer may be confirmed to have the access rights on a basis of theregistration.

This arrangement makes it possible to determine whether the externalcomputer has access rights to the robot.

(14) In the above method, the map information indicating the home of theuser may include latitude and longitude information about the home ofthe user.

With this arrangement, map information including information aboutlatitude and longitude can be used to accurately specify the location ofthe user's home.

(15) In the above method, the surrounding information about the usercollected by the robot may be acquired from the robot.

With this arrangement, the situation around the user can be specifiedaccurately.

(16) In the above method, the surrounding information about the user maybe transmitted to the external computer.

With this arrangement, the external computer can acquire surroundinginformation.

(17) A method according to another aspect is a method for controlling arobot in a robot control system including a server that communicateswith a robot of a user, the method comprising: acquiring, from anexternal computer, request information indicating a request for therobot to keep within a prescribed distance relative to a location of theuser of the robot and to collect surrounding information about the userwhen the user is outside a prescribed area including a home of the user;acquiring, from the robot, event information indicating that the user ofthe robot has moved outside the prescribed area including the home ofthe user while accompanying the robot, the event information beinggenerated by i) map information that is stored in the robot, ii) thelocation information, and iii) the confirmation information, wherein i)the map information indicates the home of the user, ii) the locationinformation indicates the location of the robot, and iii) theconfirmation information indicates that the robot is near the user on abasis of detection by the robot; and transmitting, to the robot on abasis of the event information, a command for changing a setting of therobot from a first specification to a second specification in responseto the request, the first specification enabling the robot to collectthe surrounding information about the user inside the prescribed areaincluding the home of the user, and the second specification enablingthe robot to collect the surrounding information about the user movingoutside the prescribed area including the home of the user.

With this arrangement, the robot generates event information indicatingthat the user has moved outside the home while escorted by the robot,and the event information is transmitted to the server.

(18) A method according to another aspect is a r method in a robotcontrol system including a server that communicates with a robot of auser and stores map information indicating a home of the user, themethod comprising: acquiring, from the robot, location informationindicating a location of the robot and confirmation informationindicating that the robot is near the user; outputting the mapinformation, the location information, and the confirmation informationto an external computer; acquiring, from the external computer, requestinformation indicating a request for the robot to keep within aprescribed distance relative to a location of the user and to collectsurrounding information about the user when the user of the robot isoutside a prescribed area including the home of the user, the requestinformation being acquired from the external computer in a case where itis determined at the external computer, on a basis of the mapinformation, the location information, and the confirmation information,that the user of the robot has moved outside the prescribed areaincluding the home of the user while accompanying the robot; andtransmitting, to the robot on a basis of the request, a command forchanging a setting of the robot from a first specification to a secondspecification, the first specification enabling the robot to collect thesurrounding information about the user inside the prescribed areaincluding the home of the user, and the second specification enablingthe robot to collect the surrounding information about the user movingoutside the prescribed area including the home of the user.

With this arrangement, the second server determines that the user hasmoved outside the home while escorted by the robot.

(19) In the above method, the prescribed area including the home mayinclude an area expressing a location and planar extent of the home.

According to this configuration, the location and planar extent of thehome can be used to appropriately represent the area including theuser's home.

(20) In the above method, the prescribed area including the home mayinclude an area within a prescribed distance from the home.

According to this configuration, a prescribed distance from the user'shome can be used to appropriately represent the area expressing thehome.

(21) In the above robot control method, the prescribed area expressingthe home may include an area which includes the home and which is set bythe user.

According to this configuration, the area indicating the user's home canbe set by the user.

(22) In the above method, the prescribed area including the home mayinclude an area which includes the home and which is set in the robotcontrol system.

According to this configuration, the area indicating the user's home canbe set by the robot control system.

(23) In the above robot control method, the confirmation information mayindicate that the robot has determined that the robot is near the useron a basis of a result of recognizing the user by image recognition.

With this arrangement, the presence of the user near the robot can bespecified accurately by image recognition.

(24) In the above method, the confirmation information may indicate thatthe robot has determined that the robot is near the user on a basis of aresult of acquiring, from a terminal carried by the user, currentlocation information about the terminal.

According to this configuration, the current location information of theuser's terminal can be used to accurately specify the presence of theuser near the robot.

(25) In the above method, the confirmation information may includesecond location information which indicates a location of the user fromthe robot and which is determined by the robot on a basis of a result ofrecognizing the user.

According to this configuration, a second location informationrecognized by the robot and indicating the location of the user from therobot can be used to accurately specify the presence of the user nearthe robot.

(26) In the above method, the changed setting of the robot may include asetting enabling a mobility skill provided in the robot to move at ahigher speed compared to a case where the robot is inside the home ofthe user.

According to this configuration, the robot outside the home can movefaster compared to inside the home, and therefore the user's safetyoutside the home can be ensured appropriately.

(27) In the above method, the changed setting of the robot may include asetting that turns on illumination equipment provided in the robot toilluminate the ground near the feet of the user who is moving outsidethe home of the user.

According to this configuration, the area around the user's feet isilluminated, and the safety of the user moving outside the home can beensured appropriately.

(28) In the above method, the command may include an instruction causingthe robot to output to the robot control system, after the setting ofthe robot is changed, an indication that the setting of the robot hasbeen changed.

With this arrangement, the completion of the setting change of the robotinside the robot can be confirmed by the user of the robot from theoutside of the robot.

(29) In the above method, in a case where it is confirmed that theexternal computer has access rights for making the robot perform therequest indicated by the request information, the command may betransmitted to the robot.

This arrangement makes it possible to prevent a setting change of therobot based on a remote instruction from an external source with noaccess rights to the robot.

(30) In the above method, an indication of whether the external computerhas the access rights may be registered in the server, and the externalcomputer may be confirmed to have the access rights on a basis of theregistration.

This arrangement makes it possible to determine whether the externalcomputer has access rights to the robot.

(31) In the above method, the map information indicating the home of theuser may include latitude and longitude information about the home ofthe user.

With this arrangement, map information including information aboutlatitude and longitude can be used to accurately specify the location ofthe user's home.

(32) In the above method, the surrounding information about the usercollected by the robot may be acquired from the robot.

With this arrangement, the situation around the user can be specifiedaccurately.

(33) In the above method, the surrounding information about the user maybe transmitted to the external computer.

With this arrangement, the situation around the user can be specifiedaccurately.

(34) A method according to another aspect is a method for controlling arobot in a robot control system including a server that communicateswith a robot possessed by a user, the method comprising: acquiring, fromthe robot, event information indicating that the user of the robot hasmoved outside a prescribed area including the home of the user whileaccompanying the robot, the event information being generated by therobot on a basis of i) map information that is stored in the robot, ii)location information, and iii) confirmation information, wherein i) themap information indicates the home of the user being stored in therobot, ii) location information indicates the location of the robot andiii) confirmation information indicates that the robot is near the useron a basis of detection by the robot; outputting the event informationto the external computer; acquiring, from the external computer inresponse to the event information, request information indicating arequest for the robot to keep within a prescribed distance relative to alocation of the user and to collect surrounding information about theuser when the user of the robot is outside the prescribed area includingthe home of the user; and transmitting, to the robot on a basis of therequest, a command for changing a setting of the robot from a firstspecification to a second specification, the first specificationenabling the robot to collect the surrounding information about the userinside the prescribed area including the home of the user, and thesecond specification enabling the robot to collect the surroundinginformation about the user moving outside the prescribed area includingthe home of the user.

According to the above aspect, the robot generates event informationindicating that the user has moved outside the home while escorted bythe robot, and the event information is transmitted to the first server.

(35) A method according to another aspect is a method for controlling arobot that communicates with an external computer using map informationindicating a home of a user, the method comprising: acquiring, from theexternal computer, request information indicating a request for therobot to keep within a prescribed distance relative to a location of theuser of the robot and to collect surrounding information about the userwhen the user is outside a prescribed area including the home of theuser; acquiring, on a basis of at least one internal sensor of therobot, location information indicating a location of the robot andconfirmation information indicating that the robot is near the user; andchanging a setting of the robot from a first specification to a secondspecification in response to the request in a case where it isdetermined, on a basis of the map information, the location information,and the confirmation information, that the user of the robot isaccompanying the robot outside the prescribed area including the home ofthe user, wherein the first specification enables the robot to collectthe surrounding information about the user inside the prescribed areaincluding the home of the user, and the second specification enables therobot to collect the surrounding information about the user outside theprescribed area expressing the home of the user.

According to the above aspect, the robot determines that the user hasmoved outside the home while escorted by the robot, and changes thespecifications of the robot on the basis of the determination result.

(36) A robot according to another aspect comprises: a main body; atleast one pair of legs or wheels; the at least one internal sensor; aprocessor; and a memory storing a program causing the processor toexecute the method according to (35).

The above aspect makes it possible to provide a robot that determinesthat the user has moved outside the home while escorted by the robot,and changes the specifications of the robot on the basis of thedetermination result.

(37) In the above robot, the external computer may be a communicationterminal carried by the user.

According to the above aspect, request information is acquired from thecommunication terminal carried by the user.

(38) In the above robot, the external computer may be a third-partyserver.

According to the above aspect, request information is acquired from athird-party server.

It should be noted that the present disclosure may also be implementedas a program that causes a computer to execute the characteristicfeatures included in the control method used herein, or as a system thatoperates according to the program. In addition, a computer program likethe above obviously may be distributed on a non-transitorycomputer-readable recording medium such as CD-ROM, or over acommunication network such as the Internet.

Note that the embodiments described hereinafter all illustrate specificexamples of the present disclosure. Features such as numerical values,shapes, structural elements, steps, and the ordering of steps indicatedin the following embodiments are merely examples, and are not intendedto limit the present disclosure. In addition, among the structuralelements in the following embodiments, structural elements that are notdescribed in the independent claim indicating the broadest concept aredescribed as arbitrary or optional structural elements. Furthermore, theindividual contents in all of the embodiments may also be combined.Moreover, in the following description, content denoted in parenthesesshould be interpreted as being a rewording, an exemplification, orsupplementary information regarding the immediately preceding content.

Embodiments

The Internet is expected to become even more widespread in society inthe future, with various types of sensors becoming commonplace.Accordingly, in society, information related to the internal states andactivities of individuals as well as information about a city as awhole, including features such as buildings and traffic, is anticipatedto become digitized and usable by computer systems. Digitized datarelated to individuals (personal information) is managed securely incloud servers such as information banks as big data throughcommunication networks and used for various purposes for individuals andsociety.

Such a highly information-oriented society is referred to as Society 5.0in Japan. A highly information-oriented society is one in which economicadvancement and the resolution of social issues are anticipated throughan information infrastructure (cyber-physical system) that highlyintegrates real space (physical space), which is the physical world thatsurrounds people, and virtual space (cyberspace), where computerscooperate with each other to perform various processes related tophysical space.

In such a highly information-oriented society, communication (includingthe acquisition and provision of information, and methods of expressingsuch information) and activities that an individual carries out in avariety of situations in daily life are analyzed, and by analyzing bigdata containing accumulated personal information, it is possible toprovide the individual information and services that the individualneeds via a method of communication that is considered optimal for theindividual according to the situation.

Hereinafter, close support with the daily life of an individual userwill be treated as a theme to describe a specific arrangement forincreasing a user's health and happiness on the presupposition of ahighly information-oriented society in which such a cyber-physicalsystem operates.

FIG. 1 is a block diagram illustrating an example of an overallconfiguration of an information system according to an embodiment of thepresent disclosure. The upper half of FIG. 1 illustrates cyberspacewhile the lower half illustrates physical space. On the left side of thediagram, resources related to a non-robot-providing company A arearranged, including a company A server 101 in cyberspace, and a companyA app that runs on a user's smartphone 100 in physical space. Thecompany A server 101 operates in tandem with the company A app. On theright side of the diagram, resources related to a robot-providingcompany B are arranged, including a company B server 111 in cyberspace,and a mobile device (robot 110) and a company B app that runs on thesmartphone 100 in physical space. The company B server 111 operates intandem with the robot 110 and/or the company B app. In the middle ofphysical space is the user who interacts with the company A and companyB apps installed on the smartphone 100 and with the robot 110. Thesmartphone 100, the robot 110, the company A server 101, and the companyB server 111 are communicably interconnected by a wide area networkincluding the Internet.

As illustrated in the diagram, the companies A and B have contact pointswith the user through the respective apps and the robot. The company Aonly has a contact point through the company A app on the smartphone100, which is one aspect of a customer contact point often seen intoday's world. On the other hand, the company B in the diagram not onlyhas a contact point through the company B app on the smartphone 100, butalso has a contact point through the robot 110. A company having acontact point with a user (general consumer) through an autonomousmobile device, namely the robot 110, is still unheard of aside from sometoy manufacturers, but is expected to become more widespread.

Note that a dog-like robot is adopted as an example of the robot 110herein, but the robot 110 may have a configuration based on some otherliving thing, including humans, or have an inorganic configuration thatis unlike a living thing. The configuration of the robot 110 is notlimited insofar as the robot 110 has autonomous motor skills (such asthe ability to change pose and location) and/or action skills (theability to move another object, such as pressing a button or pickingsomething up).

The information system according to an embodiment of the presentdisclosure provide higher value to users by coordinating respectivecustomer contact points, namely the apps, the robot 110, and additionalappliances and household equipment operated by the robot 110 to a higherdegree than in the past to expand the quality of its own services whileleveraging the information and skills of others. The cognitive and motorskills of the robot 110 continue to evolve on a daily basis, and if suchan all-purpose robot 110 is achieved, a framework enabling othercompanies to access the unique skills of the robot 110 should beconstructed. Such a framework would serve as the foundation for creatinga wide variety of value linkages for users, for non-robotic companiesthat provide services, and for robotic companies that provide robots.

In FIG. 1 , the company A is a security company with which the user hasa contract. The company A causes a security app provided by the companyA to be installed as the company A app on the smartphone 100, andprompts the user to establish various settings regarding securityservices provided by the company A.

The robot 110 continually collects data about the user via onboardsensors. The data about the user includes information about the user'sbiological activity and information about the user's surroundings, forexample. The biological activity information includes heart rate, bloodpressure, body temperature, activity level (calorie expenditure), numberof steps taken, posture, and exercise, for example. The surroundinginformation includes surrounding image information, informationindicating the user's location, the ambient temperature, recognitionresults regarding the surrounding space, and object recognition results,for example. The robot 110 records the collected data about the user ina memory of the robot 110. At the same time, the robot 110 periodicallyuploads the collected data about the user to the company B server 111(a).

Note that the entity that collects the biological activity informationand surrounding information is not limited to the robot 110 and may alsobe the smartphone 100, a wearable sensor (not illustrated) worn by theuser, or a sensor (not illustrated) installed in the user's home orresidential area.

The company B server 111 continually acquires data about the user fromthe robot 110. The company B server 111 allows the company A server 101having access rights to acquire the data about the user. Here,authentication settings based on user confirmation are establishedbetween the company A server 101 and the company B server 111, and thecompany A server 101 has access rights for acquiring the most recentdata held in the company B server.

The company A server 101 having access rights periodically acquires themost recent data about the user held in the company B server 111, andanalyzes the acquired data about the user. From image information aboutthe user's surroundings or temporal changes in the information about theuser's location included in the data about the user, the company Aserver 101 determines that the user has exited his or her home and istaking a walk with the robot 110. At this point, the image informationabout the user's surroundings is acquirable if the company A server 101or the company A app has a camera image access permission level of 1 orhigher, and the information about the user's location is acquirable ifthe company A server 101 or the company A app has a location sensoraccess permission level of 1. Alternatively, the company A server 101and/or company A app and the company B server 111 and/or company B appmay be set up to coordinate with each other through communication, andthe company A server 101 may be capable of detecting that the robot 110is about to guide the user on a walk of approximately 3000 steps.

In the case of determining that the user is out with the robot 110 andmoving at a walking pace, the company A server 101 transmits, to thecompany B server 111, request information for switching the securitymode of the robot 110 from “at home” to “walk”.

The security mode is one of multiple behavioral regulations(program-based robot behavioral control) set in the robot 110 by thecompany A to ensure the user's safety. When the security mode changes,the access permission levels with respect to the onboard sensors of therobot 110 are updated. This makes it easy for the company A server 101to collect necessary information. Additionally, the priority order fordetermining which actions the robot 110 should take in response to theuser and the surrounding situation also changes. Herein, the securitymode is described as being preset to switch automatically between whenthe user is at home and when the user is out, but the present disclosureis not limited thereto.

The “at home” security mode is an example of a “first specificationenabling the collection of information about the user's surroundingswhile the user is inside the prescribed area expressing the user'shome”. The “walk” security mode is an example of a “second specificationenabling the collection of information about the user's surroundingswhile the user is outside the prescribed area expressing the user'shome”.

At the timing when it is determined that the user has begun a walk, thecompany A server 101 may switch the security mode of the company A appin the smartphone 100 to “walk”. However, it is difficult for thesecurity company A to realistically ensure the user's safety while outfor a walk by switching the security mode of the company A app. Forexample, the user may not necessarily be carrying the smartphone 100when out for a walk. Furthermore, even if the user carries thesmartphone 100 and the security mode of the company A is set to “walk”,it is difficult for the company A app to realistically detect sources oftrouble and risks that the user may encounter while out for a walk, helpthe user avoid such situations, and ensure the physical safety of theuser.

In other words, there is a limit to what the company A can do as asecurity company for the user if the only daily contact point with theuser is the company A app. The autonomous robot 110 by the user's sidecan enhance the security of the user to a greater degree than an appthat does not have direct physical interaction with the real world. Forthis reason, to provide a highly effective security service through therobot 110 possessed by the user, the company A server 101 requests therobot 110 of the company B to switch to the “walk” security mode whilethe user is out for a walk.

In the information-oriented society up to the present (also referred toas Society 4.0), the security service via the company A app describedhere has been achieved. However, some dangers that occur in physicalspace would be difficult or impossible to resolve with the company Aapp. The present embodiment describes an arrangement using the robot 110which is provided with autonomous motor skills and which stays by theuser's side, whereby the company A app uses the autonomous motor skillspossessed by the robot 110 but not possessed by the company A app todesign and implement a security service that is better at keeping theuser safe. In this regard, the description of FIG. 1 will be resumed.

The company B server 111 acquiring request information from the companyA server 101 confirms whether the company A has the access rightsnecessary for the request issued from the company A server 101 to therobot 110. Here, it is assumed that various settings (the details ofwhich will be described later) regarding the security mode “walk” andaccess permission level information (the details of which will bedescribed later) indicating what is allowed when the security mode isactive are preset by the user. If the company A has access rights, thecompany B server 111 instructs the robot 110 to change to the securitymode “walk” (c). If the company A does not have access rights, thecompany B server 111 denies the request.

The robot 110 receiving the instruction for changing to the securitymode “walk” from the company B server 111 activates the security mode“walk” (d). The robot 110, having activated the security mode “walk”,escorts the user taking a walk, alerts the user taking a walk by shininga spotlight on dangerous objects, indicates to persons who pose a riskthat the user is being monitored, intimidates persons who pose a risk,and so on. Furthermore, if the user is determined to be in a state ofdanger, the robot 110 notifies the company A server 101 or the police.In this way, in the security mode “walk”, the robot 110 determinesvarious dangers that the user may possibly encounter while on a walk andensures the user's safety by avoiding the dangers and causing securitypersonnel to intervene if necessary.

Additionally, upon determining that there is a possibility that therobot 110 may be unable to ensure the user's safety, the robot 110notifies the company B server 111 with a message indicating the same.The company B server 111 forwards the notification to the company Aserver 101 (e).

The company A server 101 receiving the notification message implementssecurity by security personnel through the robot 110 on the basis of thenotification (f). Specifically, company A security personnel may listento the situation from the user and intimidate dangerous persons througha camera, microphone, and speaker of the robot 110. Furthermore, alocation sensor may be used to dispatch security personnel to the scene.

When the user finishes the walk and returns home, the company A server101 detects that the user has returned home on the basis of sensor data(such as a camera image or location information from the locationsensor) from the robot 110 acquired via the company B server 111.Accordingly, the company A server 101 transmits, to the company B server111, request information for updating the security mode of the robot 110to “at home” (g). The company B server 111 receiving the requestinformation instructs the robot 110 to update the security mode to “athome”. The robot 110 receiving the instruction activates the securitymode “at home”.

By changing the security mode from “walk” to “at home”, the robot 110autonomously changes its own behavioral regulations for ensuring theuser's safety. Specifically, the company B server 111 updates the accesspermission levels for the company A server 101 with respect to sensordata collected from the robot 110 to levels for use at home, and changesthe priority order of actions the robot 110 should take in response tothe user's state or surrounding situation to a priority order for use athome. Note that in the case where the robot 110 can confirm that thecompany A has access rights, the robot 110 may also automatically updatethe access permission levels for the company A server 101 with respectto sensor data collected by the robot 110 and the motor skills of therobot 110 to levels for use at home.

In this way, the company B server 111 automatically selects a securitymode to be provided by the company A according to the user's state orsurrounding situation. The company B server 111 may also automaticallyoptimize the disclosure levels with respect to sensor data measured bythe robot 110 that are necessary to execute security services. Forexample, to improve image recognition accuracy and raise theadmissibility of evidence if an incident occurs while the user is outfor a walk, the company B server 111 allows the company A server 101 toaccess image data captured by the camera of the robot 110 at ahigh-definition video level (permission level=3: all permitted). On theother hand, when the user is at home where there are few externalthreats, the company B server 111 allows the company A server 101 toaccess low-definition video (permission level=2: up to low-quality videopermitted) to alleviate privacy concerns.

The above description presupposes that the company A server 101 has anaccess permission level of “3” with respect to camera images. If it isassumed that the company A server 101 has an access permission level of“2” with respect to camera images, when the user is out for a walk, thecompany A server 101 would only be capable of using up to low-qualityvideo at permission level=2. If difficulties occur with securityfulfillment while the user is out for a walk, the company A server 101or the company A app may notify and/or request the user, through thecompany A app or the company B app, to raise the permission level to “3”with respect to camera images in the company B server 111.

Ensuring safety in the user's daily life is difficult to achieve withonly messages such as video information and audio information on thesmartphone 100 that serves as the customer contact point that thecompany A has with the user. By continually monitoring information aboutthe user's biological activity and information about the user'ssurroundings not through the company A app but instead through theautonomous robot 110 of the company B that stays close by the user, thecompany A can better ensure the safety of the user's daily life andprovide a higher degree of security.

In the description of FIG. 1 , the entity that determines whether or notthe user is out is the company A server 101, but the present disclosureis not limited thereto. The determining entity may also be the company Bserver 111 or the robot 110.

FIG. 2 is an illustration of the external appearance of the robot 110according to the present embodiment. In FIG. 2 , the longitudinaldirection of the robot 110 is referred to as the front-rear direction,the direction orthogonal to the walking plane of the robot 110 isreferred to as the vertical direction, and the direction orthogonal toboth the front-rear direction and the vertical direction is referred toas the left-right direction.

In FIG. 2 , a robot that moves on four legs 17 is illustrated as anexample implementation of the robot 110. The robot 110 includes a body10 and four legs 17. An illumination device 15 is disposed at the frontof the undersurface of the body 10, and an illumination device 16 isdisposed at the rear of the undersurface of the body 10. By providingthe illumination devices 15 and 16, light of sufficient intensity can beemitted in front of and behind the robot 110. The illuminationconfiguration (illumination pattern, color, blink pattern) is adjustablefor each the illumination devices 15 and 16 to illuminate dangerousobjects and make it easier for the user to be aware of the dangerousobjects. To achieve such functionality, the illumination devices 15 and16 are not lights that can emit light of a single color, but also have aprojection mapping function that projects images of any type ontosurrounding objects such as dangerous objects or streets.

A display 24 is disposed in the center of the front surface of the body10. The display 24 is a liquid crystal display (LCD) panel or an organiclight-emitting diode (OLED) panel, for example. The display 24 is mainlyused by the robot 110 to communicate with the user. As illustrated inFIG. 14 , the display 24 may display an image representing a facialexpression of the robot 110.

A display 18 is disposed in the center of the top surface of the body10, and a display 34 is disposed in the center of the rear surface ofthe body 10. The displays 18 and 34 are LCD panels or OLED panels, forexample. The displays 18 and 34 are mainly used to display messages andstates from the robot 110. For example, the displays 18 and 34 may beused to display warning information to another person in an alert areadescribed later. In this case, the displays 18 and 34 may also display alogo of the security company A that provides security services to theuser through the robot 110.

A speaker 25 is disposed in a lower part of the front surface of thebody 10, and a speaker 35 is disposed in a lower part of the rearsurface of the body 10. The speaker 25 is used by the robot 110 tocommunicate face-to-face with the user in front. The speaker 35 is usedto communicate with people approaching from behind.

An RGB camera 21, a distance measurement sensor 22, and an infraredcamera 23 are disposed on the front surface of the body 10. An RGBcamera 31, a distance measurement sensor 32, and an infrared camera 33are disposed on the rear surface of the body 10. The RGB cameras 21 and31 are used for spatial awareness and object identification. Thedistance measurement sensors 22 and 32 are used to detect the shapes ofthe surrounding space and objects, such as the shapes of dangerousobjects and uneven road surfaces. The infrared cameras 23 and 33 areused to detect people in low-light environments and the ambienttemperature distribution. By combining the RGB cameras 21 and 31, thedistance measurement sensors 22 and 32, and the infrared cameras 23 and33, the robot 110 can accurately detect the surrounding situation.

Four microphones 11, 12, 13, and 14 are disposed on the top surface ofthe body 10. By providing four microphones, the robot 110 can localizesounds.

Each leg 17 includes joints 17 a and 17 b, an upper leg 17 c, and alower leg 17 d. The joint 17 a connects the upper leg 17 c to a sidesurface of the body 10 to allow the upper leg 17 c to rotate about anaxis extending in the left-right direction. The joint 17 b attaches theupper leg 17 c and the lower leg 17 d to allow the legs to rotate aboutan axis extending in the left-right direction.

FIG. 3 is a block diagram illustrating an example of a configuration ofthe information system according to an embodiment of the presentdisclosure. The smartphone 100 includes a communication device 100 a, acomputational device 100 b, sensors 100 c, a memory 100 d, an operatingdevice 100 e, and an audio/video output device 100 f.

The communication device 100 a is a communication circuit thatcommunicates information with other computers over a network. Thecomputational device 100 b is a processor such as a CPU, for example,and performs information processing such as speech recognition, speechsynthesis, information searching, and information rendering. The sensors100 c acquire video information, audio information, and/or surroundingenvironment information. The sensors 100 c are a camera, a microphone,an acceleration sensor, an angular velocity, and a GPS sensor, forexample. The memory 100 d is flash memory, for example, and holdsvarious data. The operating device 100 e is a touch panel, for example,and receives button operations, touch operations, and the like from theuser. The audio/video output device 100 f is a display and a speaker,for example.

If the company A app and the company B app are installed, programs andnecessary data are recorded in the memory 100 d, and the programs areexecuted by the computational device 100 b.

The company A server 101 is a computer that cooperates with the companyA app installed in the smartphone 100. The company A server 101 includesa communication device 101 a, a computational device 101 b, and a memory101 c. The communication device 101 a is a communication circuit thatcommunicates information with other computers over a network. Thecomputational device 101 b is a processor such as a CPU, for example,and processes data transmitted from other computers over a network. Thememory 101 c is a solid-state drive or a hard disk drive, for example,and records information related to the company A app and the user.

The company B server 111 is a computer that cooperates with the companyB app installed in the smartphone 100. The company B server 111 includesa communication device 111 a, a computational device 111 b, and a memory111 c. The communication device 111 a is a communication circuit thatcommunicates information with other computers over a network. The memory111 c is a solid-state drive or a hard disk drive, for example, andrecords information related to the company B app, information related tothe robot 110, and information related the user. The computationaldevice 111 b processes various data transmitted from other computers.

The robot 110 includes a communication device 110 a, a computationaldevice 110 b, sensors 110 c, a memory 110 d, a moving element 110 e, anaudio/video output device 110 f, and an illumination device 110 g.

The communication device 110 a is a communication circuit thatcommunicates information with other computers over a network. Thecomputational device 110 b is a processor such as a CPU, for example.The computational device 110 b performs a process for controlling themoving element 110 e to control movement and actions by the robot 110and a process for causing the robot 110 to mechanically interact withother objects. Additionally, the computational device 110 b performs aprocessor for generating various information to be outputted from theaudio/video output device 110 f.

The sensors 110 c acquire video information, audio information, andsurrounding environment information. The sensors 110 c include the RGBcameras 21 and 31, the distance measurement sensors 22 and 32, theinfrared cameras 23 and 33, and the microphones 11 to 14, for example.

The memory 110 d is a semiconductor memory such as flash memory, forexample, and holds various data.

The moving element 110 e includes the legs 17 and an actuator that movesthe legs 17, for example. A motor is one example of an actuator. In thecase where the robot 110 has wheels instead of the legs 17, the actuatoris a motor that causes the wheels to turn. The moving element 110 ecauses the robot 110 to move, act, and mechanically interact with otherobjects.

The audio/video output device 110 f includes the speakers 25 and 35 andthe displays 18, 24, and 34, for example, and outputs audio and video.

The illumination device 110 g includes the illumination devices 15 and16 illustrated in FIG. 2 . The illumination device 110 g may include oneor more laser diodes and a microlens array or a micromirror array.Furthermore, the illumination device 110 g may include an LCD panel anda polarizer. Furthermore, the robot 110 may also be provided with anoperating device (not illustrated) that receives button operations,touch operations, and the like from the user.

Note that the wide area network connecting the company A server 101, thesmartphone 100, the robot 110, and the company B server 111 may be anyone of a mobile communication network, a satellite communicationnetwork, the Internet, a dedicated communication network, a fiber-opticnetwork, short-range wireless communication, or a combination of theabove.

FIG. 4 is a flowchart illustrating an example of a process when thecompany A and a robot cooperate. To allow the company A app or thecompany A server 101 to give the company B access to the information andskills of the robot 110 as described with reference to FIG. 1 , accessrights are set appropriately in advance. FIG. 4 illustrates an exampleof a process for setting access rights in advance.

The user uses the company B app installed in the smartphone 100 to setup the company B app to cooperate with the company A app. Specifically,the company B app acquires a unique ID of the company A app used by theuser on the basis of user input (step #1). The company B app transmits aregistration request for registering the acquired unique ID of thecompany A app with a unique ID of the company B app in the company Bserver 111 (step #2). The company B server 111 receiving theregistration request registers pairing information between the company Aapp and the company B app. In the registration process, access rightsare also registered at the same time (step #3), the access rightsindicating which skills of the robot 110 the company A is allowed toutilize, and to what degree. Details regarding access rights will bedescribed later using FIG. 5 . Pairing information including a robot IDof the robot 110 and the unique ID of the company B app is preregisteredin the company B server 111. The pairing information is registered byhaving the user input the unique ID of the robot 110 on an initial setupscreen of the company B app, for example.

The company B server 111 receiving the registration of the company A appnotifies the company A server 101 of setting information regarding theaccess rights granted to the company A app (step #4). Specifically, thecompany B server 111 notifies the company A server 101 of the settinginformation regarding the access rights in addition to the pairinginformation of the unique ID of the company A app and the unique ID ofthe company B app.

The company A server 101 registers the pairing information of the uniqueID of the company A app and the unique ID of the company B app and thesetting information regarding the access rights in the memory 101 c(step #5). The above information is used when the company A app or thecompany A server 101 utilizes the skills of the robot 110 provided bythe company B, and is used to specify the robot 110 to be utilized anddetermine whether its skills can be utilized.

The above is merely one example, and the access rights of the company Aapp or the company A server 101 with respect to robot 110 provided bythe company B may be established correctly in any way. A registrationmethod different from the above may also be used.

FIG. 5 is a table illustrating an example of relationships between typesof access rights to the robot 110 and permission levels. The robot 110is equipped with various sensors 110 c and motor skills (the movingelement 110 e). Access rights of other companies with respect to thesensors 110 c and motor skills are registered not only in the company Bserver 111, but also in the company A server 101 on the utilizing side.Hereinafter, types of access rights and their permission levels will bedescribed with respect to the sensors 110 c.

“Camera image” refers to the access rights to an image sensor (forexample, an RGB image sensor) provided in the robot 110. The imagesensor may be a sensor that is provided in a location recognized as therobot's eye from the outward appearance of the robot. As indicatedbelow, the access rights to the “camera image” are set in levels from“0”, which denotes no access rights, to “3”, which denotes unrestrictedaccess rights. For example, in response to an access request from thecompany A server 101 having a permission level of “2”, the company Bserver 111 controls the robot 110 and/or the company B server 111 toreturn low-quality video.

0: Not permitted

1: Only still images permitted

2: Up to low-quality video permitted

3: All permitted

“Distance measurement sensor” refers to the access rights to a sensor(such as a time-of-flight (ToF) sensor or LiDAR, for example) which isprovided in the robot 110 and which can measure the distance to anobject. As indicated below, the access rights to the “distancemeasurement sensor” are set to either “0”, which denotes no accessrights, or “1”, which denotes access rights. For example, in response toan access request from the company A server 101 having a permissionlevel of “1”, the company B server 111 controls the robot 110 to returndata (for example, a depth image) acquired by the distance measurementsensor.

0: Not permitted

1: Permitted

“Infrared sensor” refers to the access rights to a sensor which isprovided in the robot 110 and which can measure infrared radiation. Anear-infrared region measured by the infrared sensor is used for subjectrecognition in the dark, and a far-infrared region is used for subjecttemperature distribution or the like. As indicated below, the accessrights to the “infrared sensor” are set to either “0”, which denotes noaccess rights, or “1”, which denotes access rights. For example, inresponse to an access request from the company A server 101 having apermission level of “1”, the company B server 111 controls the robot 110to return data (for example, a thermographic image) acquired by theinfrared sensor.

0: Not permitted

1: Permitted

“Microphone audio” refers to the access rights to the microphonesprovided in the robot 110. As indicated below, the access rights to the“microphone audio” are set to either “0”, which denotes no accessrights, or “1”, which denotes access rights. For example, in response toan access request from the company A server 101 having a permissionlevel of “1”, the company B server 111 controls the robot 110 to returnaudio data acquired by the microphones.

0: Not permitted

1: Permitted

“Tactile sensor” refers to the access rights to a sensor (for example, aMEMS silicon-hair device sensor) which is provided in the robot 110 andwhich can measure a tactile sensation on the robot surface. As indicatedbelow, the access rights to the “tactile sensor” are set in levels from“0”, which denotes no access rights, to “2”, which denotes unrestrictedaccess rights. For example, in response to an access request from thecompany A server 101 having a permission level of “1”, the company Bserver 111 controls the robot 110 to return only data from a portion(for example, the head) of the robot 110 from among the data (forexample, a pressure distribution image) acquired by the tactile sensor.

0: Not permitted

1: Only partially permitted

2: All permitted

“Temperature/humidity/barometric sensor” refers to the access rights toa temperature, humidity, and barometric sensor provided in the robot110. As indicated below, the access rights to the“temperature/humidity/barometric sensor” can be set to either “0”, whichdenotes no access rights, or “1”, which denotes access rights. Forexample, in response to an access request from the company A server 101having a permission level of “1”, the company B server 111 controls therobot 110 to return data acquired by the temperature, humidity, andbarometric sensor.

0: Not permitted

1: Permitted

“Location sensor” refers to the access rights to a sensor which isprovided in the robot 110 and which measures the current location of therobot. As indicated below, the access rights to the “location sensor”are set to either “0”, which denotes no access rights, or “1”, whichdenotes access rights. For example, in response to an access requestfrom the company A server 101 having a permission level of “1”, thecompany B server 111 controls the robot 110 to return data which isacquired by the location sensor and which indicates information aboutthe current location of the robot.

0: Not permitted

1: Permitted

The description so far relates to access rights to the sensors 110 cprovided in the robot 110. Next, access rights to the “motor skills”provided in the robot 110 will be described.

“Expression-changing skills” refer to the access rights to skills forchanging the outward characteristics of a facial expression displayed bythe audio/video output device 110 f of the robot 110. In the case wherethe robot 110 has parts recognizable as a face from the outwardappearance, the expression-changing skills may involve moving the partsand changing the colors of the parts. As indicated below, the accessrights to the “expression-changing skills” are set to either “0”, whichdenotes no access rights, or “1”, which denotes access rights. Forexample, in response to an access request from the company A server 101having a permission level of “1”, the company B server 111 controls therobot 110 to change its facial expression according to a request forchanging the facial expression.

0: Not permitted

1: Permitted

“Vocalization skills” refer to the access rights to sound output skillsprovided by the audio/video output device 110 f of the robot 110. In thecase where the robot has parts recognizable as a mouth from the outwardappearance, the vocalization skills may involve moving the parts andoutputting sound from an area near the mouth. As indicated below, theaccess rights to the “vocalization skills” are set to either “0”, whichdenotes no access rights, or “1”, which denotes access rights. Forexample, in response to an access request from the company A server 101having a permission level of “1”, the company B server 111 controls therobot 110 to output sound according to sound information to bevocalized.

0: Not permitted

1: Permitted

“Pose-changing skills” refer to the access rights to skills for changingthe robot's pose provided by the moving element 110 e of the robot 110.These skills may be skills for changing the angle of the jointmechanisms in the moving element 110 e of the robot 110. However, thepose-changing skills are not intended to be skills for changing thelocation of the robot 110 itself. As indicated below, the access rightsto the “pose-changing skills” are set in levels from “0”, which denotesno access rights, to “2”, which denotes unrestricted access rights. Forexample, in response to an access request from the company A server 101having a permission level of “1”, the company B server 111 controls therobot 110 to move only the head according to the request.

0: Not permitted

1: Only head permitted

2: All permitted

“Mobility skills” refer to the access rights to skills for changinglocation provided by the moving element 110 e of the robot 110. Theseskills may be skills for changing the angle of the joint mechanisms inthe moving element 110 e of the robot 110. The mobility skills are usedto change the location of the robot 110 itself. As indicated below, theaccess rights to the “mobility skills” are set in levels from “0”, whichdenotes no access rights, to “4”, which denotes unrestricted accessrights. For example, in response to an access request from the company Aserver 101 having a permission level of “1”, the company B server 111controls the robot 110 to permit low-speed movement only in auser-approved area inside the user's home. As another example, inresponse to an access request from the company A server 101 having apermission level of “3”, the company B server 111 controls the robot 110to permit high-speed movement only in a user-approved area inside thehome and outside the home. Here, the user-approved area is a parameterthat the user sets in advance. For example, the robot 110 may be presetto not go near a certain area (such as the bath) where there would beconcerns about invading the user's privacy.

0: Not permitted

1: Low-speed movement permitted only in approved area inside home

2: Low-speed movement permitted in approved area inside/outside home

3: High-speed movement permitted in approved area inside/outside home

4: All permitted

FIG. 6 is a table illustrating an example of relationships betweensecurity modes and access rights to the robot 110 in each security mode.The table illustrates an example of how the access rights to the sensors110 c provided in the robot 110 and the access rights to the motorskills of the robot 110 are changed when the security mode is changed.Note that although the access rights are described as being changed insynchronization with the security mode, the access rights may also bechanged individually, without being synchronized with the security mode.

The table illustrates an example in which the permission levels set whenthe security mode “at home” is active are different from the permissionlevels set when the security mode is “walk” is active. Since the requestparameters regarding the sensors and motor skills needed in eachsecurity mode are different, the access rights to the sensors and motorskills are also different for different security modes. The securitymode “at home” is a security mode applied when the user and the robot110 are at home. The security mode “walk” is a security mode appliedwhen the user and the robot 110 go out for a walk.

The access rights to “camera image” are set to “2: Up to low-qualityimages permitted” for the security mode “at home”, as a setting withconsideration for the need for camera images and the user's privacy. Onthe other hand, the access rights to “camera image” are set to “3: Allpermitted” for the security mode “walk”, as a setting with considerationfor the need for camera images (improved image recognition accuracy fordangerous objects) and the admissibility of camera images as evidence.For similar reasons, the access rights to “distance measurement sensor”and “infrared sensor” are also each set to higher permission levels inthe security mode “walk” than in the security mode “at home”. Note thatthe access rights to “microphone audio”,“temperature/humidity/barometric sensor”, and “location sensor” are eachset to “1: Permitted” in both the security mode “at home” and thesecurity mode “walk”, while the access rights to “tactile sensor” areset to “0: Not permitted” in both the security mode “at home” and thesecurity mode “walk”.

With regard to “motor skills”, the access rights to “mobility skills”are set to a higher permission level in the security mode “walk” than inthe security mode “at home”. This is because a walk is taken outside thehome, and thus it is necessary to extend the range of allowed movementto outside the home. Furthermore, to enable the robot 110 to escort theuser while maintaining a fixed distance to the user, it is necessary toallow the robot 110 to move faster while on a walk compared to while athome. With regard to other motor skills, the access rights to“expression-changing skills” and “vocalization skills” are each set to“1: Permitted” in both the security mode “at home” and the security mode“walk”, while the access rights to “pose-changing skills” are set to “1:Only head permitted” in both the security mode “at home” and thesecurity mode “walk”.

“Illumination function” is a function for turning the illuminationdevice 110 g on and off. The illumination function is turned off in thesecurity mode “at home” and turned on in the security mode “walk”.

In this way, the access rights of the company A server 101 with respectto data and event information related to the robot 110 accumulated inthe company B server 111 are changed depending on the security mode. Theaccess rights of the company A server 101 in each security mode arepreset. On the other hand, to minimize the data about the user that isshared depending on the user's situation, the access rights to the dataabout the user are changed appropriately according to the security modeby the company B server 111, the robot 110, the company A app, thecompany B app, and/or the company A server 101.

FIG. 7 is a sequence diagram illustrating an example of a process by theinformation system in a case where the entity that determines whether ornot the robot 110 and the user are out is the company B server 111.

First, it is assumed that the user is at home. In addition, it isassumed that the security mode of the robot 110 is not set to either “athome” or “walk”.

(Step S1)

The robot 110 continually transmits data and event information to thecompany B server 111. The robot 110 uses the provided sensors 110 c tocontinually collect information about the user's biological activity andinformation about the user's surroundings. The robot 110 continuallytransmits data including the collected information about the user'sbiological activity and information about the user's surroundings to thecompany B server 111. The robot 110 continually generates eventinformation by analyzing the collected information about the user'sbiological activity and information about the user's surroundings, andcontinually transmits the generated event information to the company Bserver 111.

The event information is information with which the robot 110 issuesnotifications about changes in the state of the robot 110 or changes inthe state of the user detected by the robot 110. For example, the eventinformation includes “moving event information” produced upon detectingthat the user is in a moving state, “rest event information” producedupon detecting that the user is in a resting state, “sleep eventinformation” indicating that the user is in a sleeping state, “dangerevent information” produced upon detecting that the user is in a stateof danger, “low battery event information” produced upon detecting thatthe remaining battery level of the robot 110 is low, and “lost eventinformation” produced upon detecting that the robot 110 has lost trackof the user while escorting the user outside the home.

By sending event information, the robot 110 can issue notificationsregarding changes of state related to the user and the robot 110 withoutsending large quantities of sensor data. For example, if “moving eventinformation” signifying that the user has started to move is analyzedfrom a camera image obtained as video data, the processing load on theserver increases, and the load on the network bandwidth also increases.Accordingly, by using event information, more efficient cooperationbetween the robot 110 and the company B server 111 can be attained.

Furthermore, the event information includes confirmation informationindicating that the robot 110 is near the user. The confirmationinformation includes, for example, location information about thesmartphone 100 that the robot 110 has acquired from the smartphone 100,or in other words, location information about the user. Alternatively,the confirmation information may be information about a user recognitionresult obtained by the robot 110 determining whether the user appears ina camera image from an optical sensor included among the sensors 110 c,the determination being made by analyzing the camera image with faceauthentication, gait authentication, or the like. The confirmationinformation may also be information that includes location informationabout the user.

(Step S2)

The company B server 111 continually transmits or discloses the obtaineddata and event information to the company A server 101 in the categoryof the access rights set for the company A server 101, and therebyshares the data and event information with the company A server 101.

(Steps S3, S4)

The smartphone 100 sets a security mode in accordance with input fromthe user. Setting information indicating the set content of the securitymode includes the information displayed in FIG. 23 or 24 describedlater, for example. For example, the setting information includessecurity mode parameters. The security mode parameters includeparameters of the security mode “at home” and parameters of the securitymode “walk”. The parameters of the security mode “walk” are parametersthat change the security mode from “at home” to “walk”. The parametersof the security mode “at home” are parameters that change the securitymode from “walk” to “at home”. For example, when the security mode ischanged to “walk”, the setting information may include information withwhich the robot 110 notifies the user of the change. The notificationmay be the display of information on the face of the robot 110 and/or apart of the body 10, the display of a logo of the security company, theoutput of speech announcing the change, or the output of a sound logo ofthe security company. Furthermore, the notification may also be acombination of the above.

(Step S5)

The smartphone 100 transmits the setting information and requestinformation to the company A server 101. With respect to the securitymode “walk”, for example, the request information is informationstipulating a request for causing the robot 110 to execute a code ofconduct (a program, or an order of priority for actions) that causes thecomputational device 110 b to control the moving element 110 e of therobot 110 to escort the user outside the home, that is, outside a setat-home area, while keeping within a prescribed distance relative to theuser, and at the same time causes the robot 110 to collect informationabout the user's surroundings using the sensors 110 c. With respect tothe security mode “at home”, for example, the request information isinformation stipulating a request for causing the robot 110 to execute acode of conduct that causes the computational device 110 b to controlthe moving element 110 e and the sensors 110 c of the robot 110 to keepwithin a prescribed distance relative to the user to monitor the user'ssafety inside the home, that is, inside a set at-home area.

(Step S6)

The company A server 101 receiving the setting information transmits thesetting information and the request information to the company B server111.

(Step S7)

The company B server 111 registers the received setting information andrequest information in the memory 111 c.

(Step S8)

On the basis of the data and event information continually transmittedfrom the robot 110 and map information, the company B server 111confirms whether the user satisfies the parameters of one of thesecurity modes.

(Step S9)

If it is determined that the user satisfies the parameters of thesecurity mode “at home”, or if it is determined that the user satisfiesthe parameters of the security mode “at home” the most, the company Bserver 111 transmits an instruction to the robot 110 for changing to thesecurity mode “at home”.

(Step S10)

The robot 110 receiving the instruction for the security mode “at home”notifies the user about the start of the security mode “at home”.Details about the notification will be described later.

(Step S11)

The robot 110 operates in the security mode “at home”. The security modecontinues until a new security mode instruction is received.

(Step S12)

The company B server 111 detects that the security mode has been changedto “at home” according to a response from the robot 110 (notillustrated), and transmits the result of the change to the securitymode “at home” to the company A server 101.

(Step S13)

The user goes out with the robot 110 and starts a walk.

(Step S14)

Sensing results from the robot 110 satisfy the parameters of thesecurity mode “walk”, and therefore the company B server 111 changes thesecurity mode to “walk”.

(Step S15)

The company B server 111 transmits an instruction for changing thesecurity mode to “walk” to the robot 110. The instruction is an exampleof a “command for changing the setting of the robot from a firstspecification to a second specification”. The instruction includes acommand causing the robot 110 to output an indication that the securitymode has been changed after changing the security mode setting in therobot 110.

(Step S16)

The robot 110 receiving the instruction for the security mode “walk”notifies the user about the start of the security mode “walk”. Detailsabout the notification will be described later.

(Step S17)

The robot 110 continually operates in the security mode “walk”.

(Step S18)

The company B server 111 detects that the robot 110 has changed thesecurity mode “walk” according to a response from the robot 110 (notillustrated), and transmits the result of the change to the securitymode “walk” to the company A server 101.

In FIG. 7 , the company A server 101 is an example of an externalcomputer. In FIG. 7 , the smartphone 100 (or the company A app or thecompany B app executed thereon) may also transmit the settinginformation and the request information to the company B server 111. Inthis case, the smartphone 100 is an example of an external computer.

In FIG. 7 , the confirmation information is location information aboutthe user, but this is merely an example. For example, the confirmationinformation may also be information indicating that the robot 110 isdetermined to be near the user. In this case, the robot 110 maydetermine whether the robot 110 is near the user on the basis of arecognition result regarding an image of the user acquired using thesensors 110 c, or determine whether the robot 110 is near the user onthe basis of information about the current location of the smartphone100 acquired from the smartphone 100.

FIG. 8 is a diagram illustrating an example of a process by the companyB server 111 and the robot 110 corresponding to FIG. 7 .

(Step S201)

The robot 110 uses the sensors 110 c to sense the state of the user andthe surroundings. Accordingly, information about the user's biologicalactivity and information about the user's surroundings are obtained.

(Step S202)

The robot 110 transmits data including the biological information aboutthe user and the information about the user's surroundings, and eventinformation generated on the basis of the biological information aboutthe user and the information about the user's surroundings, to thecompany B server 111.

(Step S203)

The company B server 111 receives the data and event information.

(Step S204)

The company B server 111 determines whether the security mode should bechanged according to whether the data and event information receivedfrom the robot 110 satisfy the parameters of one of the security modes.For example, in the case where the currently set security mode is “athome” and the data and event information received from the robot 110satisfy the parameters of the security mode “walk”, the company B server111 determines that it is necessary to change the security mode to“walk”. As another example, in the case where the currently set securitymode is “walk” and the data and event information received from therobot 110 satisfy the parameters of the security mode “walk”, thecompany B server 111 determines that it is not necessary to change thesecurity mode. As another example, in the case where the currently setsecurity mode is “walk” and the data and event information received fromthe robot 110 do not satisfy the parameters of the security mode “walk”but do satisfy the parameters of the security mode “at home”, thecompany B server 111 determines that it is necessary to change thesecurity mode to “at home”. As another example, in the case where thecurrently set security mode is “at home” and the data and eventinformation received from the robot 110 satisfy the parameters of thesecurity mode “at home”, the company B server 111 determines that it isnot necessary to change the security mode from “at home”.

The company B server 111 compares the location information about therobot 110 included in the data and the location information about theuser included in the confirmation information of the event informationto the at-home area obtained from the map information, and if it isdetermined that the robot 110 and the user are both outside the at-homearea, the company B server 111 may determine that the user (that is, thedata and event information received from the robot 110) satisfies theparameters of the security mode “walk”. On the other hand, in the casewhere the robot 110 and the user are both at home, the company B server111 may determine that the user (that is, the data and event informationreceived from the robot 110) does not satisfy the parameters of thesecurity mode “walk” but does satisfy the parameters of the securitymode “at home”. The map information is stored in the memory 111 c andincludes information about the latitude and longitude of the home. Theat-home area is an example of a prescribed area expressing the home.

Note that in the case where the confirmation information is the resultof a determination by the robot 110 indicating that the user is near therobot 110, the company B server 111 may determine that the user (thatis, the data and event information received from the robot 110)satisfies the parameters of the security mode “at home” if the locationinformation about the robot 110 is located in the at-home area.Conversely, it may be determined that the parameters of the securitymode “walk” are satisfied if the location information about the robot110 is located outside the at-home area.

(Step S205)

If it is determined that a change of security mode is necessary (stepS205, YES), the company B server 111 advances the process to step S206.On the other hand, if it is determined that a change of security mode isnot necessary (step S205, NO), the company B server 111 ends theprocess.

(Step S206)

The company B server 111 transmits an instruction for changing thesecurity mode to the robot 110.

(Step S207)

The robot 110 receiving the instruction changes the security mode. Forexample, the robot 110 changes the security mode from “at home” to“walk”.

(Step S208)

The robot 110 transmits the result of changing the security mode to thecompany B server 111.

(Step S209)

The company B server 111 receives the result of changing the securitymode.

(Step S210)

The robot 110 notifies the user about the start of the changed securitymode. As described above, the company B server 111 uses the data andevent information from the robot 110 to successively determine thesecurity mode, and instructs the robot 110 to execute the security mode.

FIG. 9 is a diagram illustrating an example of a scenario in which thesecurity mode is switched from “at home” to “walk”. For example, anat-home area 802 is a circular region of fixed radius centered on thelocation of a home 801. The area outside the at-home area 802 is anoutside area 806 (walk area). If the robot 110 and the user are in theat-home area 802, the security mode “at home” is set, and if the robot110 and the user are in the outside area 806, the security mode “walk”is set.

If the user and the robot 110 move from the at-home area 802 to theoutside area 806, the security mode is changed to “walk”. In this case,the robot 110 may notify the user about the change to the security mode“walk” by using the audio/video output device 100 f to output soundinformation such as words, a barking sound, and a sound logo. Oneexample of the sound information is a message such as “I'll protect youon your walk! *woof*”. The robot 110 may also notify the user about thechange to the security mode “walk” by displaying image information of aprescribed mark, such as a logo of the security company, on the displays18, 24, and 34. The robot 110 may also notify the user about the changeto the security mode “walk” with a specific behavior, such as walking ontwo legs. Also, if a display is provided in a location corresponding tothe eyes, the robot 110 may also notify the user about the change to thesecurity mode “walk” by changing the appearance of the eyes. Throughthese notifications, the user can be informed about the change in thesecurity mode of the robot 110 which is difficult to discern from theoutward appearance of the robot 110.

FIGS. 10, 11, 12, and 13 are diagrams that each illustrate an example ofthe at-home area 802. In FIGS. 10 to 13 , the region inside the thicklines is the at-home area 802. The area where the user and the robot 110are, the area being a parameter for switching the security mode of therobot 110, is one of the following or a combination thereof.

(1) The at-home area 802 in FIG. 10 is a circular region with a radius804 centered on a spot 800. The spot 800 is location informationacquired by the GPS of the smartphone 100 inside home premises 803. Inthis case, for example, the at-home area 802 is prescribed by latitudeand longitude information about the spot 800 and information about theradius 804.

(2) The at-home area 802 in FIG. 11 is the region surrounded by theoutline of the home 801. The outline shape of the home 801 is acquiredfrom map information, for example.

(3) The at-home area 802 in FIG. 12 is the region surrounded by theoutline of the home premises 803. The outline of the home premises 803is acquired from map information.

(4) The at-home area 802 in FIG. 13 is a region of any shape that theuser designates on a map. For example, in a map image displayed on thesmartphone 100, the inside of an outline 805 of the home premises 803traced by the user is set as the at-home area 802.

(5) Otherwise, the at-home area 802 may be set on the basis of thelocation information of the spot 800 and a setting rule designated bythe information system. The location information of the spot 800 islocation information acquired by the GPS of the smartphone 100, forexample. In this case, a region prescribed by a setting rule based onlatitude and longitude information about the spot 800 is set as theat-home area 802. The setting rule may also be a circle like the oneillustrated in FIG. 10 .

(6) The information system may also set the at-home area 802automatically using address information about the user's home and mapinformation including the home 801 and/or the home premises 803. In thiscase, the at-home area 802 is set using home address informationdesignated by the user and map information including the home 801 and/orthe home premises 803 linked to the address information.

In all of the above cases (1) to (6), all areas other than the at-homearea 802 may be treated as the outside area 806. With this arrangement,there is a clear boundary between the at-home area 802 and the outsidearea 806, and the security mode of the robot 110 can be changedaccurately depending on the current location of the user and the robot110.

The setting information about the at-home area 802 is recorded andmanaged in one or more locations from among the company A server 101,the company B server 111, and the memory 110 d of the robot 110, forexample. The setting information may also be information indicating theoutline of the at-home area 802.

In all of the above cases (1) to (6), the setting information about theat-home area 802 may also be confirmed and set by the user through thesmartphone 100. The setting information is acquired through the companyA app, the company B app, or software by another company.

FIGS. 14 to 16 are diagrams illustrating examples of notifications thatthe robot 110 issues to the user. FIG. 14 illustrates a notificationissued when the security mode is changed to “at home”. In the example ofFIG. 14 , the robot 110 displays images 1401 of left and right eyes withpupils having small circular glowing portions. FIG. 15 illustrates anotification issued when the security mode of the robot 110 is changedto “walk”. In the example of FIG. 15 , the robot 110 displays images1401 of left and right eyes with pupils having long, narrow verticallines. FIG. 16 illustrates a notification issued by the robot 110 whenspecific event information is generated. In the example of FIG. 16 , therobot 110 displays images 1401 of left and right eyes with pupils havinga crescent shape, as though the eyelids are half-closed. One example ofthe event information is low battery event information. Low batteryrefers to a state in which the remaining battery level of the robot 110has fallen to a prescribed level or lower.

According to the above, the user can look at the image informationdisplayed in the portion of the robot 110 corresponding to the eyes ofthe audio/video output device 110 f, and thereby easily confirm whetherthe current mode of the robot 110 is the security mode “at home” or“walk” and also confirm whether event information such as “low battery”is occurring in the robot 110. These images 1401 are predeterminedaccording to the types of security modes and the types of eventinformation. Note that the images 1401 are displayed on the display 24(FIG. 2 ), for example.

Next, an arrangement in which the entity that determines whether or notthe robot 110 and the user are out is the company A server 101 (anexample of a second server) will be described. FIG. 17 is a sequencediagram illustrating an example of a process by an information system ina case where the entity that determines whether or not the robot 110 andthe user are out is the company A server 101.

In FIG. 17 , processes which are the same as FIG. 7 are denoted by thesame signs and may be omitted from the following description.

(Step S301)

The user goes out with the robot 110 and starts a walk.

(Step S302)

The company A server 101 acquires the most recent data and eventinformation about the user and the robot 110 through the company Bserver 111, and thereby continually determines the necessity of changingthe security mode and security personnel intervention while monitoringthe user and the robot 110. For example, from data acquired by thesensors 110 c (location information about the robot 110), the company Aserver 101 detects that the robot 110 is out, from a camera imagecaptured by the robot 110 (or confirmation information indicating thesame), the company A server 101 detects that the user is near the robot110 and that the user is moving at a walking pace (that is, the user iswalking). Alternatively, the company A server 101 may detect that theuser has started a walk outside the at-home area from a notification of“walk start event information” issued by the robot 110 to the company Bserver 111. Note that in the above case, the company A server 101acquires the data and event information acquired by the robot 110through the company B server 111, but the company A server 101 mayadditionally acquire map information, including at-home areainformation, through the company B server 111.

(Step S303)

The company A server 101 analyzes the acquired data and eventinformation, detects that the user has started a walk with the robot110, and transmits request information for changing the security modefrom “at home” to “walk” to the company B server 111. Like in FIG. 7 ,the company A server 101 may transmit request information for changingthe security mode from “at home” to “walk” if it is determined that theuser (that is, the data and event information received from the 110)satisfies the parameters of the security mode.

The access rights that the company A server 101 has with respect to thesensors 110 c and the motor skills of the robot 110 necessary forguarding the user through the robot 110 are different between thesecurity mode “at home” and the security mode “walk”. For this reason,the company A server 101 transmits request information for changing thesecurity mode from “at home” to “walk” to the company B server 111. Byswitching the security mode, the access rights with respect to thesensors 110 c and the motor skills are changed automatically.

(Step S304)

The company B server 111 receiving the request information transmits, tothe robot 110, an instruction for changing the security mode of therobot 110 to “walk”. The security modes “at home” and “walk” usable bythe company A server 101 are preregistered in the company B server 111,similarly to the access rights to the sensors and motor skills.

(Step S305)

The robot 110 receiving the instruction for changing to the securitymode “walk” changes its security mode to “walk”. Thereafter, as long asan instruction for changing to a security mode other than “walk” is notreceived from the company B server 111, the computational device 110 bof the robot 110 continues to control the moving element 110 e and thesensors 110 c such that the robot 110 operates on the basis of the codeof conduct specified by the security mode “walk”.

(Step S306)

Even after changing the security mode to “walk”, the robot 110 continuesto transmit data acquired using the sensors 110 c and event informationgenerated on the basis of the data to the company B server 111.

While the user continues the walk, the robot 110 continues to guard theuser according to the code of conduct stipulated by the security mode“walk”. How the robot 110 escorts the user and how the robot 110responds to dangers that the user encounters are preset by the userusing the company A app. Details will be described later.

(Step S307)

A danger to the user occurs. A danger refers to a situation in whichthere is an obstacle in the user's forward direction and a situation inwhich another person is approaching the user, for example.

(Step S308)

The company A server 101 receives data and event information from thecompany B server 111 and determines the necessity of changing thesecurity mode “walk” and the necessity of security personnelintervention. In addition to the data and event information, the companyA server 101 may also receive map information including at-home areainformation from the company B server 111.

(Step S309)

The company A server 101 determines the necessity of intervention bysecurity personnel for ensuring the user's safety on the basis of thedata and event information. The determination may also be made bysecurity personnel (a human being) checking the camera image, that is,video data captured by the robot 110, for example. Alternatively, thecompany A server 101 may use image recognition technology to calculate adanger level of the user from the camera image, and determine thenecessity of intervention by security personnel from the calculateddanger level. Alternatively, the company A server 101 uses speechrecognition technology or a technology for analyzing the user's emotionsfrom speech to analyze speech information from the user acquired by therobot 110 and determine the danger level of the user, and determine thenecessity of intervention by security personnel from the determineddanger level. Alternatively, if image recognition technology is used todetermine that a danger to the user has occurred, the company A server101 may also transmit an alert indicating the occurrence to an externalterminal operated by security personnel. In this case, the securitypersonnel may check the alert through the external terminal and mobilizeto go to the scene.

There are two stages of intervention by the company A securitypersonnel. The first stage is remote security intervention. Securitypersonnel do not go to the scene, but instead take measures for avoidingdanger by using the sensors 110 c and the audio/video output device 110f of the robot 110 to talk to the user at the scene, a person near theuser, or a person attempting to harm the user. The second stage issecurity intervention at the scene. Security personnel are dispatched tothe scene and take action for avoiding danger to the user at the scene.Both stages of security intervention are security services executed whenthe company A server 101 or the company A security personnel havedetermined that it would be difficult to ensure the user's safety withthe robot 110 alone.

If the user is in a state of danger, the robot 110 may use the sensors110 c to sense the user and the situation around the user to enablelater inspection of the scene, and saved the sensed data in the companyA server 101, the company B server 111, and/or the memory 111 c of therobot 110. In addition, the company A server 101, the company B server111, and the robot 110 may also share the sensed data together withcontrol information that disables deletion of the data for a fixedperiod. The company A server 101, the company B server 111, and/or therobot 110 may also calculate a hash value for proving that the senseddata has not been tampered with, and save the calculated hash value intwo or more locations (for example, a company C server and the company Bserver).

(Step S310)

The user avoids danger thanks to the security service provided by thecompany A, and returns home safely. At this point, the walk ends.

(Step S311)

The company A server 101 receives data and event information from thecompany B server 111 and determines the necessity of changing thesecurity mode “walk” and the necessity of security personnelintervention. In addition to the data and event information, the companyA server 101 may also receive map information including at-home areainformation from the company B server 111.

(Step S312)

The company A server 101 detects, from the user's return home, that theuser (that is, the data and event information received from the robot110) does not satisfy the parameters of the security mode “walk” butinstead corresponds to the parameters of the security mode “at home”,and transmits request information for changing the security mode from“walk” to “at home” to the company B server 111.

(Step S313)

The company B server 111 receiving the request information outputs, tothe robot 110, an instruction for changing the security mode from “walk”to “at home”.

(Step S314)

The robot 110 receiving the instruction for updating to the securitymode “at home” from the company B server 111 changes its security modeto “at home”. Thereafter, as long as an instruction for changing to anew security mode is not received from the company B server 111, thecomputational device 110 b of the robot 110 continues to control themoving element 110 e and the sensors 110 c such that the robot 110operates according to the code of conduct specified by the security mode“at home”.

(Steps S315, S316)

Even after updating the security mode to “at home”, the robot 110continues to transmit data and event information to the company B server111, similarly to step S1. Similarly to step S302, the company A server101 continually acquires the most recent data and event information fromthe company B server 111, and continually determines the necessity ofchanging the security mode and security personnel intervention. Inaddition to the data and event information, the company A server 101 mayalso receive map information including at-home area information from thecompany B server 111.

In FIG. 17 , the company A server 101 is an example of an externalcomputer and a second server. In FIG. 17 , the smartphone 100 (or thecompany A app or the company B app executed thereon) may also transmitthe setting information and the request information to the company Bserver 111. In this case, the smartphone 100 is an example of anexternal computer. In this case, the smartphone 100 may acquire data andevent information from the company B server 111 and determine whether ornot to change the smartphone on the basis of the acquired data and eventinformation. In FIG. 17 , the event information includes confirmationinformation, similarly to FIG. 7 . The details of the confirmationinformation are the same as FIG. 7 .

FIG. 18 is a flowchart illustrating an example of a process by thecompany A server 101, the company B server 111, and the robot 110corresponding to FIG. 17 .

(Step S401)

The robot 110 transmits data and event information to the company Bserver 111.

(Step S402)

The company B server 111 receives the data and event information fromthe robot 110.

(Step S403)

The company A server 101 receives data and event information from thecompany B server 111, and also receives the set at-home area informationfrom the memory 101 c or the company A app. In addition to the data andevent information, the company A server 101 may also receive mapinformation including at-home area information from the company B server111.

(Step S403)

The company A server 101 determines whether the security mode should bechanged through comparison with the parameters of the security modes onthe basis of the user (that is, the data and event information receivedfrom the robot 110). The details of this process are the same as stepS204. If it is determined that a change of security mode is necessary(step S403, YES), the company A server 101 advances the process to stepS404. On the other hand, if it is determined that a change of securitymode is not necessary (step S403, NO; in other words, the currentsecurity mode is to be continued), the company A server 101 ends theprocess.

(Step S404)

The company A server 101 transmits an instruction for changing thesecurity mode to the company B server 111.

(Step S405)

The company B server 111 receiving the instruction checks the accessrights of the company A server, and then transmits the instruction tothe robot 110.

(Step S406)

The robot 110 receiving the instruction changes the security mode.

(Step S407)

The robot 110 transmits the result of changing the security mode to thecompany B server 111.

(Step S408)

The company B server 111 receives the result of the change.

(Step S409)

The company B server 111 transmits the result of the change to thecompany A server 101.

(Step S410)

The company A server 101 receives the result of the change.

(Step S411)

The robot 110 notifies the user about the start of the newly changedsecurity mode. The details of the notification are the same as thearrangement described above with reference to FIGS. 9, 14, and 15 . Inthe case where the company A server 101 is the entity that determinesthe security mode, the security mode is changed according to the aboveprocess.

Next, an arrangement in which the entity that determines whether or notthe robot 110 and the user are out and also determines the security modeis the robot 110 will be described. FIG. 19 is a sequence diagramillustrating an example of an information processing system in a casewhere the entity that determines whether or not the robot 110 and theuser are out and also determines the security mode is the robot 110. InFIG. 19 , processes which are the same as FIG. 7 are denoted by the samesigns and may be omitted from the following description.

In FIG. 19 , the smartphone 100 and the robot 110 and paired in advancethrough a means of communication such as Wi-Fi®, Bluetooth®, or acellular communication network (4G, 5G).

(Step S501)

The smartphone 100 (or the company A app or the company B app executedthereon) transmits the setting information and the request informationto the robot 110. The details of the setting information and the requestinformation are the same as FIG. 7 .

(Step S502)

On the basis of the data and event information continually transmittedfrom the sensors 110 c and the preset setting information and requestinformation, the robot 110 confirms whether the user satisfies theparameters of one of the security modes. At this point, the usersatisfies the parameters of the security mode “at home”, and thereforethe robot 110 changes its security mode to “at home”. In addition to thedata and event information, the company A server 101 may also receivemap information including at-home area information from the company Bserver 111.

(Step S503)

The robot 110 notifies the user about the start of the security mode “athome”. This is the same as step S10.

(Step S504)

The robot 110 continually operates in the security mode “at home”. Thisis the same as step S11.

(Step S505)

Since the security mode has been changed to “at home”, the robot 110transmits the result of changing to the security mode “at home” to thecompany B server 111.

(Step S506)

Since the security mode has been changed to “at home”, the robot 110transmits the result of changing to the security mode “at home” to thesmartphone 100 (or the company A app or the company B app executedthereon).

(Step S507)

The user goes out with the robot 110 and starts a walk.

(Step S508)

The user satisfies the parameters of the security mode “walk”, andtherefore the robot 110 changes its security mode to “walk”.

(Step S509)

The robot 110 notifies the user about the start of the security mode“walk”. The details of the notification are the same as in FIGS. 14 to16 described above.

(Step S510)

The robot 110 continually operates in the security mode “walk”.

(Step S511)

Since the security mode has been changed to “walk”, the robot 110transmits the result of changing to the security mode “walk” to thecompany B server 111.

(Step S512)

Since the security mode has been changed to “walk”, the robot 110transmits the result of changing to the security mode “walk” to thesmartphone 100.

In FIG. 19 , the smartphone 100 (or the company A app or the company Bapp executed thereon) is an example of an external computer. In FIG. 19, the smartphone 100, the company A app, or the company B app may alsotransmit the setting information and the request information to therobot 110 by going through (or not going through) the company B server111. Alternatively, the setting information and the request informationmay be transmitted from the company A server 101 to the robot 110 bygoing through (or not going through) the company B server 111. In thiscase, the company A server 101 is an example of an external computer. InFIG. 19 , the event information includes confirmation information,similarly to FIG. 7 . The details of the confirmation information arethe same as FIG. 7 .

FIG. 20 is a flowchart illustrating an example of a process by the robot110 corresponding to FIG. 19 .

(Step S601)

The robot 110 uses the sensors 110 c to sense the state of the user andthe surroundings. Accordingly, information about the user's biologicalactivity and information about the user's surroundings are obtained.

(Step S602)

The robot 110 determines whether the security mode should be changedaccording to whether the user (that is, the data and event informationreceived from the robot 110) satisfies the parameters of one of thesecurity modes. The details of this determination are the same as stepS204.

(Step S603)

If it is determined that a change of security mode is necessary (stepS603, YES), the robot 110 advances the process to step S604. On theother hand, if it is determined that a change of security mode is notnecessary (step S603, NO), the robot 110 ends the process. In this case,the current security mode is continued.

(Step S604)

The robot 110 changes its security mode. For example, the robot 110changes from the current security mode “at home” to the security mode“walk”. Obviously, the reverse change may also be performed in somecases.

(Step S605)

The robot 110 notifies the user about the start of the changed securitymode. In this case, a notification is issued like in FIG. 14 or 15described above.

FIG. 21 is a flowchart illustrating an example of a process for updatinga necessary security level in the robot 110.

The necessary security level refers to an indicator obtained bycomprehensively evaluating the degree to which the robot 110 needs toensure the user's safety according to the state of the user andsituation surrounding the user.

The necessary security level is expressed as a value from 0 (minimum) to3 (maximum) as a result of determining the danger level. The specificactions of the robot 110 are determined according to this value incombination with the currently applied security mode.

(Step S6001)

The robot 110 acquires data indicating the state of the user and sensingdata indicating the situation surrounding the user, the data beingsensed by the sensors 110 c. The sensing data is, for example, a cameraimage, a depth image, an infrared image, microphone audio, tactilesensor values, temperature/humidity/barometric sensor values, and/orlocation information. The robot 110 may also acquire sensing data sensedby external sensors over a network. The external sensors are, forexample, sensors included in a wearable device worn by the user, camerasinstalled at street corners, and information such as weather forecastinformation and road traffic information corresponding to the locationinformation about the robot 110, such information being acquirable overthe Internet.

(Step S6002)

The robot 110 determines the necessary security level of the user fromthe danger level of the user obtained by analyzing the acquired sensingdata. The following describes processes according to the determinedvalue of the necessary security level.

(Step S6003)

If it is determined from the information from the sensors 110 c that nodangerous objects or approachers exist near the user, the computationaldevice 110 b of the robot 110 determines that the necessary securitylevel is 0 (minimum danger level). The computational device 110 b of therobot 110 controls the moving element 110 e, the illumination device 110g, and the like of the robot 110 to perform operations for supportingthe user on a walk, and returns the process to step S6001. For example,the robot 110 may use the illumination device 110 g to display, ahead ofthe user in the forward direction, a target number of steps for thewalk, a target calorie expenditure for the walk, a target walking timefor the walk, the number of steps taken since beginning the walk, thenumber of steps taken today, the remaining number of steps until today'starget number of steps is reached, the user's heart rate, the user'scalorie expenditure, the elapsed time of the walk, and/or the predictedtime remaining on the walk.

(Step S6004)

If it is determined from the information from the sensors 110 c that adangerous object exists near the user, the computational device 110 b ofthe robot 110 determines that the necessary security level is 1 (lowdanger level). The computational device 110 b of the robot 110 controlsthe moving element 110 e, the illumination device 110 g, and the like ofthe robot 110 to perform supporting operations for preventing injury tothe user on the walk, performs processes such as alerting the user byilluminating the detected danger and choosing a route that bypasses thedanger, and returns the process to step S6001. For example, the robot110 may use the illumination device 110 g to shine light on a dangerousobject near the user and alert the user about the object. A dangerousobject refers to an uneven road surface or step of prescribed size, anobstacle ahead in the forward direction, and the like.

Instead of, or in addition to, illuminating a dangerous object withlight, if a dangerous object is detected, the robot 110 may also controlthe moving element 110 e to move in front of or beside the user to guidethe user such that the user walks on a safe route keeping a certaindistance from the dangerous object.

(Step S6005)

If it is determined from the information from the sensors 110 c that adangerous person (or other) exists near the user, the computationaldevice 110 b of the robot 110 sets the necessary security level to 2(high danger level). The computational device 110 b of the robot 110controls the moving element 110 e and the like of the robot 110 toperform operations for ensuring the safety of the user on a walk, andreturns the process to step S6001. For example, the robot 110 uses theillumination device 110 g to display, around the user, informationindicating that the user is being guarded by the robot 110. Theinformation indicating that the user is being guarded is, for example, aring of light surrounding the user, a log of the security company A, andthe characters “under guard”.

Alternatively, the robot 110 may use the audio/video output device 110 fto output sounds that alert or intimidate a suspicious person. The alertnotifies the suspicious person that the company A is guarding the user,for example. The intimidation notifies the suspicious person that he orshe could be immediately reported to the company A guarding the user,for example.

Alternatively, the robot 110 may control the moving element 110 e suchthat the robot 110 positions itself between the user and the suspiciousperson to maintain a certain distance between the user and thesuspicious person.

(Step S6006)

If it is determined from the information from the sensors 110 c that adangerous person (or other) has not been successfully removed from thevicinity of the user, and/or if the robot 110 is unable to keep thedangerous person (or other) a certain distance from the user viasecurity measures using the audio/video output device 110 f, the movingelement 110 e, and the illumination device 110 g of the robot 110, thecomputational device 110 b of the robot 110 sets the necessary securitylevel to 3 (highest danger level). In this case, the user is in adangerous situation requiring intervention by (human) securitypersonnel, and therefore the computational device 110 b of the robot 110transmits, to the company A server 101, a signal requesting company Asecurity personnel for ensuring the safety of the user on the walk, andreturns the process to step S6001. For example, the robot 110 may usethe sensors 110 c and the audio/video output device 100 f to enablesecurity personnel to talk with the suspicious person directly andensure the user's safety. If the user is in a more dangerous situation,or if the state of danger persists even after performing remote securityby the security personnel, the robot 110 may transmit, to the company Aserver 101, a dispatch request for dispatching company A securitypersonnel to the scene. Alternatively, the robot 110 may report to thepolice. The report may include information informing the police of thelocation of the scene and the situation at the scene by using audio,text, and/or camera images.

The necessary security level may be determined by an external computerexternal to the robot 110, such as any of the company B server 111, thecompany A server 101, the smartphone 100, or the company A app orcompany B app executed on the smartphone 100, for example. In this case,the robot 110 may transmit the sensing data from the sensors 110 c, theevent information obtained by having the computational device 110 bprocess the sensing data, and information about the necessary securitylevel currently implemented by the robot 110 to the external computerover a network, thereby causing the communication device 110 a of therobot 110 to receive the necessary security level from the externalcomputer.

FIG. 22 is a diagram illustrating an example of a home screen 2100displayed on the smartphone 100 after launching the company A app. Thehome screen 2100 includes a button 2101 labeled “set at-home security”,a button 2102 labeled “set security while out (on a walk) with company Brobot”, and a button 2103 labeled “emergency contact number”.

The button 2101 is a button for establishing various settings related tothe security provided by the robot 110 when the user is at home with therobot 110. The button 2102 is a button for establishing various settingsrelated to the security provided by the robot 110 when the user is outon a walk together with the robot 110. The button 2103 is a button forspecifying the contact number of someone to be notified about the user'sdanger if an emergency occurs, or for defining the conditions underwhich such a notification is issued.

If the user launches the company A app installed on the smartphone 100and performs a touch operation or speech operation on the button 2102, asettings screen 2200 in FIG. 23 is displayed.

FIG. 23 is a diagram illustrating an example of the settings screen2200. The settings screen 2200 is a screen for setting how the robot 110escorts the user when the user goes on a walk with the robot, whatmeasures are to be taken by the robot in response to certain kinds ofdangers, what kind of supplementary information about the walk is to bereported to the user, and the like.

A setting field 2201 labeled “escort home direction” is a field by whichthe user uses a pull-down menu to set the home direction of the robot110 when the robot 110 escorts the user. The home direction is thedirection of a home position. The home position is the spot where therobot 110 is positioned relative to the user when escorting the user.The home direction is the direction, relative to the forward directionof the user, in which the home position of the robot 110 is located.

For example, the pull-down menu of the setting field 2201 is configuredsuch that one of the following operations is selectable: “in front”which indicates in front of the user; “diagonally ahead to the right”which is located 45 degrees clockwise from “front”; “right side” whichis located another 45 degrees clockwise; “diagonally behind to theright” which is located another 45 degrees clockwise; “behind” which islocated another 45 degrees clockwise; “diagonally behind to the left”which is located another 45 degrees clockwise; “left side” which islocated another 45 degrees clockwise; and “diagonally ahead to the left”which is located another 45 degrees clockwise. In the diagram, the userhas set “diagonally ahead to the right”, and therefore the robot 110moves with the direction “diagonally ahead to the right” of the user asthe home direction. To maintain the designated home direction, the robot110 uses a camera image or the like acquired by the sensors 110 c tosense the user's location and forward direction, and controls the movingelement 110 e to adjust its own position on the basis of the sensingresult.

A setting field 2202 labeled “escort home distance” is a field by whichthe user uses a slider to set a home distance. The home distance is thedistance between the robot 110 and the user when escorting the user. Forexample, the slider is configured such that the home distance can be setin the range from 1 m to 5 m. In the diagram, the user has set adistance of approximately 2 m, and therefore the robot 110 moves bycontrolling the moving element 110 e to basically maintain a distanceabout 2 m away from the user. To maintain the designated home distance,the robot 110 uses a camera image, depth image, and the like acquired bythe sensors 110 c to sense the user's location, and corrects its ownposition.

A setting field 2203 labeled “illuminate nearby ground” is a setting forturning on or turning off a nearby ground illumination function by whichthe robot 110 illuminates the road surface from the user's feet in theforward direction of the user. In the diagram, the user has turned thefunction “on”, and therefore the robot 110 controls the beam directionof the illumination device 110 g to illuminate the nearby ground at thefeet of the user on a walk. Note that in the case where the illuminationdevice 110 g is not capable of changing the beam direction, the robot110 may control the beam direction by using the moving element 110 e toalter the inclination of a portion of the robot 110 that includes theillumination device 110 g. To maintain illumination indicating theforward direction while on a walk, the robot 110 may use a camera imageor the like acquired by the sensors 110 c to sense the user's locationand forward direction, and adjust the illumination device 110 g and/orthe inclination of the above portion on the basis of the sensing result.

On the other hand, if the illumination function is turned “off”, therobot 110 does not activate the function for illuminating the nearbyground at the user's feet.

By continually illuminating the nearby ground, nearby persons can bemade aware that the robot 110 is ensuring the user's safety. As aresult, by turning “on” the nearby ground illumination function, aneffect of keeping the user out of dangerous situations with suspiciouspersons can also be anticipated.

A setting field 2204 labeled “alert user to dangerous objects” is afield for turning on or turning off an alert user to dangerous objectsfunction by which the robot 110 illuminates (with a spotlight) and/orcalls out with sound a dangerous object in front of the user. In thediagram, the user has turned the function “on”, and therefore the robot110 shines a spotlight on a dangerous object in the forward direction(ahead) of the user on a walk, or outputs sound alerting the user to thedangerous object. The robot 110 may use a camera image, depth image, andthe like acquired by the sensors 110 c to detect a dangerous object inthe forward direction of the user and nearby, and adjust theillumination device 110 g and/or the inclination of a portion includingthe illumination device 110 g to illuminate the dangerous object. Therobot 110 may also use the audio/video output device 110 f to inform theuser that there is a dangerous object. If the robot 110 detects adangerous object, the robot 110 may move from the home position to aposition in front of the user and guide the user on a safe route thatthe user should take. In this case, the robot 110 may return to the homeposition again after the user has passed the dangerous object.

On the other hand, if the function for alerting the user to dangerousobject” is turned “off”, the robot 110 does not alert the user todangerous objects.

Dangerous objects include, for example, objects of a prescribed size orlarger, uneven road surfaces, level differences in the road surface, andobstructions in the forward direction of the user. Alternatively,dangerous objects may be cars, bicycles, and the like approaching theuser, for example.

A setting field 2205 labeled “alert surroundings” is a field for turningon or turning off a function by which the robot 110 gives notice ordemonstrates, in an easily understood way, the presence of the user tomoving bodies including people around the user. Besides people, themoving bodies include cars and bicycles ridden by people and objects orvehicles that move autonomously by computer control, for example. Withthis function, nearby people can be made aware that the user is beingguarded by the robot 110. In the diagram, the user has turned “on” thefunction for alerting the surroundings, and therefore the robot 110gives notice or demonstrates, in an easily understood way, the presenceof the user to moving bodies near or approaching the user on a walk.

The robot 110 uses a camera image, depth image, and the like acquired bythe sensors 110 c to detect moving bodies near or approaching the user.If necessary, the robot 110 uses the illumination device 110 g todisplay information on the road surface or the like to a detected movingbody, illuminates the user so that the user is perceived more easily,and uses the audio/video output device 110 f to call out to movingbodies with sound or speech.

Also, if autonomous moving bodies such as other robots, self-drivingcars, and drones approach the user, the robot 110 may use thecommunication device 110 a to broadcast a wireless communication signal(for example, a beacon) indicating information about the currentlocation of the user and/or the presence of the robot 110. Thisarrangement makes it possible to avoid contact with the user and therobot 110.

On the other hand, if the function for alerting the surroundings isturned “off”, the robot 110 does not alert moving bodies near orapproaching the user.

A setting field 2206 labeled “alert radius” is a parameter related to“alert surroundings”. In the diagram, the alert radius has been set toapproximately 5 m, and therefore persons or moving bodies within 5 mfrom the user are alerted to the presence of the user. If the “alertsurroundings” function is “off”, the alert radius is disabled.

A setting field 2207 labeled “report steps” is a setting for turning onor turning off a function by which the robot 110 notifies the user ofinformation about the user's steps. In the diagram, the function isturned “on”, and therefore the robot 110 notifies the user ofinformation about the steps taken by the user on a walk. For example,the robot 110 uses the audio/video output device 110 f to output speechnotifying the user about the number of steps, in units of 1000 steps,that the user has taken since the start of the walk. The robot 110 mayalso notify the user of step information by using the illuminationdevice 110 g to display, on the road surface, the number of steps thatthe user has taken today.

On the other hand, if the function is turned “off”, the robot 110 doesnot notify the user of step information while on a walk.

A setting field 2208 labeled “report biological information” is asetting for turning on or turning off a function by which the robot 110notifies the user of information about the user's biological activity.In the diagram, the function is turned “off”, and therefore the robot110 does not notify the user of information about the user's biologicalactivity while on a walk. If the function is turned “on”, similarly to“report steps” above, the robot 110 uses the audio/video output device110 f and the illumination device 110 g to notify the user of biologicalactivity information such as the user's heart rate, blood pressure, bodytemperature, and activity level (calorie expenditure).

In this way, through the settings screen 2200, the user is able toestablish presets to his or her liking with regard to various operationsthat the robot 110 performs while on a walk. With this arrangement, theuser can use the robot 110 to go on a walk, which is associated with acertain degree of risk, more safely. Moreover, the user gains a sense ofrelief knowing that he or she can go for a walk safely, therebyincreasing opportunities for the user to maintain his or her health andattain mental stability.

FIG. 24 is a diagram illustrating a settings screen 2300 in anotherexample according to the embodiment. A setting field 2301 labeled“radius of at-home area” is a field for setting the radius 804illustrated in FIG. 10 . In this field, the user can use a slider toinput the radius 804 in the range from 10 m to 100 m. In this example,the radius is set to approximately 30 m. Consequently, the range withinapproximately 30 m from the center of the home is set as the at-homearea 802.

A setting field 2302 labeled “escort home position” is a field forsetting the home position. The setting field 2302 displays iconsrepresenting the robot 110 and the user as though looking down on therobot 110 and the user from overhead. The user sets the home position bymoving the icon of the robot 110 with a finger Y. Note that the settingfield 2302 may also set the home position by prompting the user to inputthe home direction and the home distance described above. The user mayalso set the home position by saying “run on my left side” or byperforming a gesture input, for example.

The robot 110 may also use the sensors 110 c to sense a state of theuser, such as the user walking, jogging, or cycling, and adjust the homeposition according to the sensed state of the user. The robot 110 mayalso adjust the home position dynamically on the basis of thesurrounding situation, such as the type of road such as a sidewalk, thewidth of the road, the presence or absence of level differences in theroad, and the amount of nearby pedestrian and automotive traffic.

For example, if the sensors 110 c detect that nearby people are notpresent, the robot 110 escorts the user diagonally ahead to the right atthe home position set by the user. On the other hand, if there are morepeople around the user or if the user passes by another person, therobot 110 may escort the user by temporarily moving to a position thatdoes not obstruct the user, the surrounding people, or the passerby. Theposition that does not obstruct is a position directly in front of ordirectly behind the user in the forward direction of the user, forexample.

In the case where the robot 110 uses image recognition to track theuser's body, it is natural for the robot 110 to be positioned behind theuser and follow the user. However, there are demerits to staying behindthe user from the standpoint of ensuring the user's safety, such as theinability to provide illumination in the forward direction of the userand the inability to detect dangerous objects ahead. Accordingly, thehome position may be set in an area not behind the user, and from thishome position, the robot 110 may provide illumination in the forwarddirection of the user and detect dangerous objects ahead.

Note that a settings screen according to the embodiment may also includesetting fields from both the settings screen 2200 and the settingsscreen 2300 in a non-redundant way.

The settings screens 2200 and 2300 are examples of a graphical userinterface displayed on the screen of a communication terminal.

A setting field 2303 is a field for setting the image design of the eyesof the robot 110 while the security mode “walk” is in effect. An eyeimage design like the one described with reference to FIG. 15 is sethere.

A setting field 2304 is a field for turning on or turning off thecontrol of the behavior performed using the moving element 110 e tonotify the user about the code of conduct of the robot 110 or that thesecurity mode has been switched. A setting field 2305 is a setting forturning on or turning off the control for causing the audio/video outputdevice 110 f to display the logo of the security company A. A settingfield 2306 is a setting for turning on or turning off the control forcausing the audio/video output device 110 f to output the sound logo ofthe security company A. By turning these settings on, it is possible tocause the robot 110 to control the moving element 110 e and theaudio/video output device 110 f to perform a specific behavior andoutput a specific sound like in FIG. 9 .

FIGS. 25, 26, and 27 are diagrams illustrating a first example of thesecurity mode “walk”. FIG. 25 illustrates an example of “illuminatenearby ground” described above. In the example of FIG. 25 , the robot110 is positioned behind the user. In FIG. 25 , a robot position 131which is the center position of the robot 110 is indicated by a small“x” symbol, while a user position 130 which is the center position ofthe user is indicated by a large “x” symbol. In FIG. 25 , the ellipticalarea extending from the user's feet forward in a movement direction D1of the user is a nearby ground area 120 where the robot 110 illuminatesthe ground at the user's feet with the illumination device 110 g.

The long-axis direction of the nearby ground area 120 points in themovement direction D1 of the user. By illuminating the nearby groundarea 120 in this way, the robot 110 can enable the user to easily seethe state of the road surface. Accordingly, the user is able to walksafely with a reduced risk of falling and being injured while on a walk.Note that in the case where a measurement result regarding ambientbrightness obtained by the sensors 110 c indicates a brightly lit placeor daytime, the robot 110 may turn “off” the “illuminate nearby ground”function even if the function is set to “on”. Also, in the case where“low battery event information” is issued, the robot 110 may conservebattery power by narrowing the range and lowering the brightness of thenearby ground area 120.

FIG. 26 illustrates an example of “alert user to dangerous objects”described above. In this example, the robot 110 has detected a pylonindicating a restricted area as a dangerous object in the movementdirection D1 of the user, and is shining a spotlight on the dangerousobject. In this scenario, the necessary security level is 1. A dangerarea 122 is the area where the robot 110 is illuminating the recognizeddangerous object. By causing the illumination device 110 g to illuminatethe danger area 122, the user can easily see that there is a dangerousobject. In the example of FIG. 26 , the danger area 122 is a circle ofapproximately the same size as the dangerous object.

The robot 110 may use a technology such as simultaneous localization andmapping (SLAM) to generate a map of the user's surroundings from sensingdata acquired by the sensors 110 c. The robot 110 may then use the mapto continuously calculate the user position 130, the robot position 131,the positions of dangerous objects, and the like. The robot 110 maycalculate the movement direction D1 of the user from the direction thatthe user is facing, a history of the user position 130, and the like,and determine as a dangerous object an object that is within aprescribed distance to the left or right of the movement direction D1and also within a prescribed distance ahead in the movement direction D1from the user position 130.

In addition to the nearby ground area 120, the robot 110 may alsoilluminate a travel route area 121 indicating a travel route recommendedto the user. The travel route area 121 has an arrow shape extending inthe forward direction from the nearby ground area 120. The arrow shapehas a shape conforming to the travel route recommended for avoidingdangerous objects. With this arrangement, the recommended travel routeis illuminated and the user can walk along the recommended travel route.The robot 110 may calculate the recommended travel route on the basis ofthe user position 130, the robot position 131, a dangerous objectposition 133, and information about the path width. A path searchalgorithm for self-driving that calculates a travel route for avoidingdangerous objects is adopted as the process for calculating therecommended travel route, for example.

In this way, in the case where “alert user to dangerous objects” isperformed, the robot 110 needs to illuminate not only the nearby groundarea 120 but also the travel route area 121 or the danger area 122. Forthis reason, when the robot 110 illuminates these areas, the robot 110may move away from the designated home position and to a position wherethe areas are easily illuminated by the audio/video output device 110 f,and assume a pose with which the areas are easily illuminated. In theexample in FIG. 27 , the robot 110 moves to an emission position 132diagonally behind to the left of the user and in front of the homeposition 134 to illuminate a broad range ahead of the user. If there isan area to be illuminated in front, the robot 110 may also move to aposition in the front of the user in the area from the left side to theright side.

In the scenario in FIG. 27 , the robot 110 has calculated the homeposition 134 designated in advance on the basis of sensing resultsregarding the user position 130 and the movement direction D1 of theuser, and is escorting the user while positioning itself at the homeposition 134. The robot 110 senses the surroundings, detects the pylondiagonally ahead to the right as a dangerous object, and bends theleading end of the nearby ground area 120 to the left to guide the useron a travel route that avoids the dangerous object.

Furthermore, the robot 110 is illuminating a triangular danger area 122fitted to the shape of the pylon treated as a dangerous object. Thedanger area 122 may be a different color from the nearby ground area120. For example, the nearby ground area 120 may be white and the dangerarea 122 may be red. The danger area 122 may also blink to promote userawareness. The blink interval may also vary depending on the distancebetween the dangerous object and the user. For example, the blinkinterval may be shortened as the dangerous object and the user getcloser to one another. With this arrangement, the user can easilyperceive the dangerous object.

The shape, color, and blink pattern of the danger area 122 are notlimited to the above, and any appearance may be adopted insofar as theuser can distinguish a dangerous object from the surroundingenvironment. For example, one or more of the shape, color, and blinkpattern of the danger area 122 may be controlled. Additionally, one ormore of the color and blink pattern of the nearby ground area 120 mayalso be controlled. In this case, the nearby ground area 120 iscontrolled differently from the danger area 122.

In FIG. 27 , since the user is positioned between the robot 110 and thedangerous object, the robot 110 located at the robot position 131 insidethe home position 134 cannot irradiate the nearby ground area 120 andthe danger area 122 at the same time, or cannot provide irradiationeffectively. Otherwise, there are various possible reasons why the robot110 would have difficulty irradiating the nearby ground area 120 and thedanger area 122 at the same time, such as in the case where theirradiation range of the illumination device 110 g of the robot 110 isonly in the forward direction of the robot 110, or the case where theillumination device 110 g has a structure that cannot radiate light inthe opposite direction of the forward direction.

In this way, if it is necessary to irradiate the nearby ground area 120and the danger area 122 at the same time, but not all of the areas to beirradiated can be irradiated from the home position 134, the robot 110controls the moving element 110 e to temporarily move from the robotposition 131 to an emission position 132 suitable for irradiating thenearby ground area 120 and the danger area 122 at the same time. Theemission position 132 may be determined as a target position that therobot 110 attempting to change its position relative to the user shouldreach at a prescribed time.

In the case where the robot 110 newly detects a dangerous object at theemission position 132 after moving and thus detects a plurality ofdangerous objects in the movement direction D1 of the user, the robot110 may determine a new emission position 132 from which a plurality ofdanger areas 122 irradiating the plurality of dangerous objects and thenearby ground area 120 can be irradiated at the same time, andtemporarily move to the newly determined emission position 132.

In FIG. 27 , the robot 110 determines an arc-shaped emission area 123having a certain thickness diagonally behind to the left from the userposition 130, determines the emission position 132 inside the emissionarea 123, moves to the emission position 132, and irradiates the nearbyground area 120 and the danger area 122. The emission area 123 is anarea calculated on the basis of the user position 130, the robotposition 131, the dangerous object position 133, and the irradiationrange of the illumination device 110 g, and is an area from which boththe danger area 122 and the user position 130 can be irradiated. Theemission area 123 may be calculated by the computational device 110 busing, for example, a calculation model that accepts the user position130, the movement direction D1 (or a movement vector indicating both themovement direction and the movement speed), the dangerous objectposition 133, and the irradiation range of the illumination device 110 gas input and outputs the emission area 123.

The emission position 132 is the position inside the emission area 123that is closest to the robot position 131, for example. However, this ismerely an example, the emission position 132 may be any position insidethe emission area 123. Note that the robot 110 may also calculate theemission position 132 on the basis of the user position 130, the robotposition 131, the dangerous object position 133, and the irradiationrange of the illumination device 110 g, without calculating the emissionarea 123. If the irradiation of a dangerous object is no longernecessary, the robot 110 moves back into the home position 134 andcontinues irradiating the nearby ground area 120 only.

FIG. 28 is a diagram illustrating the behavior of the robot 110 whenmoving from the home position 134 to the emission position 132. At atime t0, the robot 110 is not detecting a dangerous object, or isdetecting a dangerous object but the distance from the user to thedangerous object is a prescribed distance or longer. Consequently, therobot 110 escorts the user while maintaining the robot position 131inside the home position 134.

At a time t0+dt, the robot 110 detects a dangerous object, and toirradiate both the dangerous object position 133 and the nearby groundarea 120, the robot 110 starts moving from the current robot position131 to the closest emission position 132 inside the emission area 123from which both areas can be irradiated effectively. At the same time,the robot 110 directs the long-axis direction of the nearby ground area120 to the left to guide the user on a travel route that avoids thedangerous object. Furthermore, the robot 110 also starts radiating thedanger area 122 toward the dangerous object position 133. However, sincethe robot 110 has not reached the emission position 132, only a portionof the dangerous object position 133 is irradiated.

At a time t0+2*dt, the robot 110 has moved to the emission position 132.The robot 110 continues detecting the user position 130, the movementdirection D1, and the dangerous object position 133, and continuesupdating the nearby ground area 120 and the danger area 122. The nearbyground area 120 has a shape with the long-axis direction inclined to theleft so that the user will avoid the dangerous object. Since the robot110 has moved to the emission position 132, the danger area 122 isirradiating the entirety of the dangerous object. With this arrangement,the user can recognize the dangerous object and the modified travelroute, turn his or her forward direction to the left, and achieve a safewalk that avoids the dangerous object.

At a time t0+3*dt, the user passes by the dangerous object. The robot110 escorts the user while continuing to detect the user position 130,the movement direction D1, the robot position 131, and the dangerousobject position 133, and continuing to update the emission area 123suitable for irradiating the nearby ground area 120 and the danger area122 and also update the emission position 132 therein. With thisarrangement, the user recognizes the dangerous object and the modifiedtravel route, and passes by the dangerous object.

As illustrated in FIG. 28 , the robot 110 continues to irradiate thenearby ground area 120 and the danger area 122 even while moving fromthe home position 134 to the emission position 132.

FIG. 29 is a flowchart illustrating an example of a process by the robot110 in a first example of the security mode “walk”.

(Step S801)

The robot 110 uses the sensors 110 c to acquire surrounding information.The surrounding information includes the user position 130, the movementdirection D1, the robot position 131, and the dangerous object position133.

For example, the robot 110 may detect a dangerous object near the userby inputting a camera image acquired by the sensors 110 c into aclassifier that specifies the type of an object, and detect thedangerous object position 133 by extracting, from a depth image acquiredby the sensors 110 c, the distance to the detected dangerous object fromthe robot 110.

Similarly, the robot 110 may detect the user by inputting a camera imageacquired by the sensors 110 c into a classifier, and detect the userposition 130 by extracting, from a depth image acquired by the sensors110 c, the distance to the detected user from the robot 110.

For example, the robot 110 may detect the robot position 131 fromlocation information acquired by a location sensor of the sensors 110 c.

For example, the robot 110 may detect the movement direction D1 from thedirection the user is facing, which is detected from a history of theuser position 130 or a camera image.

(Step S802)

The robot 110 detects whether there is a dangerous object toward themovement direction D1 of the user. If a dangerous object is not detectedtoward the movement direction D1 of the user (step S802, NO), theprocess proceeds to step S803, whereas if a dangerous object is detectedtoward the movement direction D1 of the user (step S802, YES), theprocess proceeds to step S804. Toward the movement direction D1 of theuser refers to ahead of the user. Ahead of the user includes themovement direction D1 of the user and directions included inside anangle range extending 90 degrees to the left and right from the movementdirection D1 at the center.

For example, in the case where the detected dangerous object position133 is located inside a prescribed area ahead of the user, the robot 110may determine that there is a dangerous object in the forward directionof the user.

(Step S803)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 0.

The robot 110 determines the emission position 132 suitable forirradiating the nearby ground area 120. The emission position 132 inthis case is basically the home position 134. However, if it isdifficult to irradiate the nearby ground area 120 from the preset homeposition 134, the emission position 132 may also be a sub-home positionwhich is an escort position different from the home position 134.

(Step S804)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 1.

The robot 110 determines the emission position 132 suitable forirradiating the nearby ground area 120 and the danger area 122 on thebasis of the user position 130, the movement direction D1, the robotposition 131, the dangerous object position 133, and the irradiationrange of the illumination device 110 g. For example, the robot 110determines the emission area 123 by inputting the user position 130, themovement direction D1, the robot position 131, the dangerous objectposition 133, and the irradiation range of the illumination device 110 ginto the calculation model described above. Additionally, if the homeposition 134 or the sub-home position is not inside the determinedemission area 123, the robot 110 calculates the position inside theemission area 123 that is closest to the robot position 131 as theemission position 132. The state in which the home position 134 and thesub-home position are not inside the emission area 123 corresponds to anexample in which the emission position 132 is different from the currentrelative position of the robot 110. If the home position or the sub-homeposition is inside the emission area 123, the home position or thesub-home position is treated as the emission position 132.

(Step S805)

The robot 110 controls the moving element 110 e to move to the emissionposition 132.

(Step S806)

The robot 110 controls the moving element 110 e to escort the user fromthe emission position 132. The robot 110 continues to calculate the userposition 130, the robot position 131, the movement direction D1, and thedangerous object position 133.

(Step S807)

If the security mode “walk” is not stopped (step S807, NO), the robot110 returns the process to step S801 and continues the process from stepS801. If the security mode “walk” is stopped (step S807, YES), the robot110 ends the process. The security mode “walk” is stopped when the userreturns to the at-home area, for example.

The flowchart in FIG. 29 may also include a process for irradiating thenearby ground area 120 only if the “illuminate nearby ground” functiondescribed with reference to FIG. 23 is set to “on”. The flowchart inFIG. 29 may also include a process for irradiating the danger area 122only if the “alert user to dangerous objects” function described withreference to FIG. 23 is set to “on”.

FIG. 30 is a diagram illustrating a second example of the security mode“walk”. In the second example of the security mode “walk”, if adangerous object is located in the movement direction D1 of the user,the robot 110 sets the robot position 131 to a lead position 135 in adirection ahead of the user in the movement direction (a directionwithin 90 degrees to the left or right of the movement direction D1) andalso different from the movement direction D1, moves to the leadposition 135, and escorts the user from the lead position 135. The leadposition 135 may be determined as a target position that the robot 110attempting to change its position relative to the user should reach at aprescribed time.

In the scenario in FIG. 30 , the robot 110 has calculated the homeposition 134 set in advance on the basis of sensing results regardingthe user position 130 and the movement direction D1, and is escortingthe user while controlling the moving element 110 e to position itselfinside the home position 134.

The robot 110 senses the surroundings and detects an uneven road surfacediagonally ahead to the right as a dangerous object. In this case, therobot 110 calculates the lead position 135 for guiding the user on atravel route recommended to avoid the dangerous object. For example, therobot 110 calculates a recommended travel route using the path searchalgorithm described above, sets a lead area 124 along the calculatedrecommended travel route, and calculates the position inside the leadarea 124 that is closest to the robot position 131 as the lead position135.

The lead area 124 is an area of prescribed size along the recommendedtravel route, located a prescribed distance ahead of the user position130. Here, the shape of the lead area 124 is rectangular, but this ismerely an example, and the shape may also be circular, elliptical, orthe like. The lead area 124 may also be an area where the illuminationdevice 110 g of the robot 110 can additionally irradiate a dangerousobject. The lead position 135 is the position inside the lead area 124that is closest to the robot position 131, but this is merely anexample, and the lead area 124 may be any position inside the lead area124. The robot 110 continues to irradiate the nearby ground area 120with light even while moving from the robot position 131 to the leadposition 135.

The robot 110 temporarily leaves the home position 134 and escorts theuser from the lead position 135 until the user passes the dangerousobject. In this example, the dangerous object is located diagonallyahead to the right of the user, and therefore the recommended travelroute is pointed diagonally ahead to the left of the user.

Furthermore, in the example of FIG. 30 , to guide the user on therecommended travel route, the robot 110 inclines the nearby ground area120 such that the long-axis direction of the nearby ground area 120points slightly leftward with respect to the movement direction D1

In this way, the robot 110 moves to the lead position 135 and irradiatesthe nearby ground area 120 indicating the travel route that the usershould follow while guiding the user on the travel route. With thisarrangement, the user can avoid dangers such as falling due to thedangerous object. The user follows the guidance provided by the robot110, and after passing the dangerous object, the robot 110 returnsinside the original home position 134 or the sub-home position, andcontinues to irradiate the nearby ground area 120.

FIG. 31 is a flowchart illustrating an example of a process by the robot110 in the second example of the security mode “walk”.

Steps S901, S902, and S907 are the same as steps S801, S802, and S807 inFIG. 29 .

(Step S903)

The computational device 110 b of the robot 110 sets the necessarysecurity level too.

The robot 110 sets a lead position 135 suitable for when no dangerousobject exists. The lead position 135 is basically the home position 134.However, the lead position 135 may also be the sub-home position if itis difficult to irradiate the nearby ground area 120 from the presethome position 134.

(Step S904)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 1.

The robot 110 sets a lead position 135 suitable for when a dangerousobject exists. For example, the lead position 135 suitable for when adangerous object exists is the position inside the lead area 124 that isclosest to the robot position 131.

(Step S905)

The robot 110 controls the moving element 110 e to move to the leadposition 135.

(Step S906)

The robot 110 controls the moving element 110 e to escort the user fromthe lead position 135.

The flowchart in FIG. 31 may also include a process for irradiating thenearby ground area 120 only if the “illuminate nearby ground” functiondescribed with reference to FIG. 23 is set to “on”. The flowchart inFIG. 31 may also include a process for irradiating the danger area 122only if the “alert user to dangerous objects” function described withreference to FIG. 23 is set to “on”.

Next, a third example of the security mode “walk” will be described.FIGS. 32, 33 , and 34 are diagrams illustrating the third example of thesecurity mode “walk”. Note that FIG. 34 is a different example fromFIGS. 32 and 33 . In the third example of the security mode “walk”, amoving body exists near the user, and if the moving body comes withinthe range of a prescribed distance from the user, the robot 110 sets aguard position 143 and escorts the user from the guard position 143. Theguard position 143 is a position where the robot 110 is between the userand the moving body. The moving body is a person approaching the user,for example. The guard position 143 may be determined as a targetposition that the robot 110 attempting to change its position relativeto the user should reach at a prescribed time.

In FIG. 32 , the large x symbol near the bottom illustrates a personposition 140, which is the position of a person approaching the userfrom behind. In the diagram, the distance 146 between the user position130 and the person position 140 is greater than an alert radius 142 setby the user, and the person is not alerted.

The robot 110 senses the situation surrounding the user by using acamera image, depth image, infrared image, microphone audio, and thelike acquired by the sensors 110 c. The robot 110 detects that a personis approaching the user from behind. In this case, the robot 110 setsthe guard position 143 (FIG. 33 ) and controls the moving element 110 eto start moving to the guard position 143. The above movement isperformed to display information for alerting the person at a positionon the road surface between the user and the person approaching theuser, and also to maintain the distance 146.

In FIG. 33 , the person has further approached the user from behind andthe distance 146 is now less than the alert radius 142, and thereforethe robot 110 alerts the person. In this scenario, the necessarysecurity level is 2.

The robot 110, having reached the guard position 143 and inserted itselfbetween the user position 130 and the person position 140, uses theillumination device 110 g to display alert information at a position onthe road surface between the user and the person. In this case, an arc144 centered on the user position 130 is displayed. With thisarrangement, arc 144 can be seen by the person, and the person can bealerted not to approach the user any further. Additionally, the robot110 displays, near the arc 144, a message to the person. Here, themessage “PASS ON THE RIGHT PLEASE” advising the person to pass by on theuser's right side is displayed. At the same time, the nearby ground area120 is changed such that the long axis points in a direction diagonallyto the left from the movement direction D1. With this arrangement, theuser is guided slightly to the left. Following the guidance, the usershifts to the left. As a result, the person passes the user more easily.In this case, although the nearby ground area 120 is changed, the travelroute area 121 (FIG. 26 ) expressing the travel route of the user is notdisplayed. The display area of the arc 144 and the message is an exampleof a notification area between the user and a moving body.

Even if the person has approached the user with malicious intent, it ispossible to inform the person that the user's safety is being ensured bythe smart and autonomous robot 110, and it can be expected that theperson will not harm the user. If the person further approaches theuser, or if it is detected that the user feels afraid or agitated, therobot 110 may change the information displayed on the road surface as analert to a stronger expression or a warning. The user's fear oragitation is detectable by applying an emotion estimation process to acamera image and microphone audio acquired by the sensors 110 c. Inaddition to the display on the road surface, the robot 110 may also usethe audio/video output device 110 f to output sound or speech that callsout to the person, or sound or speech in a loud voice that intimidatesthe person. Furthermore, if a dangerous state (necessary securitylevel=3) is determined, the robot 110 may also transmit a report to thecompany A server 101.

FIGS. 32 and 33 can be applied similarly to the case where a moving bodyother than a person approaches the user. Moving bodies other than aperson include, for example, human-driven cars, bicycles, and flightvehicles (drones), and computer-controlled flight vehicles and groundvehicles that move autonomously.

In FIG. 32 , instead of the alert radius 142, an alert area 141 behindthe user position 130 may also be adopted. For example, the robot 110may issue the alert described above if a person advancing in the sameforward direction as the user is detected inside the alert area 141.This arrangement makes it possible to alert only a person who has a highlikelihood of contacting the user if he or she continues to advanceforward. As a result, there is a reduction in alerting persons with alow likelihood of harming the user. The alert area 141 is an area ofprescribed size a prescribed distance behind the user position 130.

In FIG. 34 , the robot 110 has calculated the home position 134 on thebasis of sensing results regarding the user position 130 and themovement direction D1, and is escorting the user from the home position134. The robot 110 senses the surroundings and detects a personapproaching the user from diagonally behind to the left. The robot 110calculates a travel route for creating distance from the user, andalters the shape of the nearby ground area 120 to guide the user on thetravel route. In this case, a travel route guiding the user diagonallyahead to the right is calculated, and the nearby ground area 120 has ashape with the leading end pointed diagonally ahead to the right toguide the user on the travel route. With this arrangement, the persondiagonally behind to the left can be kept away from the user.

In the diagram, the robot 110 is positioned at the home position 134just beside the user position 130, and therefore cannot insert itselfbetween the user and the person while also irradiating the nearby groundarea 120.

Accordingly, if the robot 110 determines that it is necessary to insertitself between the person approaching the user and the user to issue analert, the robot 110 controls the moving element 110 e to move towardthe guard position 143, which is inside a guard area 145 suitable forgetting between the person and the user, and from which the robot 110can irradiate the nearby ground area 120 at the user's feet.

The robot 110 may determine that an alert is necessary if a personsatisfying an alert condition is present near the user. The alertcondition is, for example, the presence of a person approaching a circleprescribed by the alert radius 142, the presence of a person inside thecircle prescribed by the alert radius 142, the presence of a person nearthe alert area 141, the presence of a person inside the alert area 141,and the user uttering speech or performing a gesture for requestingprotection.

The robot 110 may determine that a person approaching the circleprescribed by the alert radius 142 or the alert area 141 is present if amovement direction D2 of the person within a prescribed distance ispointed toward the circle prescribed by the alert radius 142 or thealert area 141. The robot 110 may determine that a person is presentinside the circle prescribed by the alert radius 142 or the alert area141 if the person position 140 is inside the circle prescribed by thealert radius 142 or the alert area 141.

The robot 110, having determined that the person satisfies the alertcondition, sets the guard position 143 and temporarily moves to theguard position 143. The robot 110 may also continue to irradiate thenearby ground area 120 with light even while moving from the homeposition 134 or the sub-home position to the guard position 143.

When an alert is no longer necessary, the robot 110 leaves the guardposition 143, returns to the home position 134, and continues to escortthe user from the home position 134. Note that the robot 110 may alsoreturn to the home position 134 if the robot 110 detects speech or agesture from the user indicating that protection is unnecessary.

In the example of FIG. 34 , the position which is inside the thickarc-shaped guard area 145 positioned midway between the user position130 and the person position 140 and which is closest to the robotposition 131 is set as the guard position 143.

Note that if it is determined that an alert is necessary, the robot 110may prioritize moving to the guard position 143 over irradiating thenearby ground area 120. In this case, the nearby ground area 120 may bepartially missing or darkened, but by having the robot 110 insert itselfbetween the person approaching the user and the user, it is possible tomake the person aware that the user is being guarded by the robot 110,thereby raising the user's safety further.

Furthermore, the robot 110 may display, on the road surface, a warning147 that advises the approaching person to keep a certain distance fromthe user. The warning 147 may be a mark or characters for guiding thewalking route of the person, for example. The mark is the mark of thesecurity company A, for example. In this case, the robot 110 may move toa position inside the midway area between the user position 130 and theperson position 140, and from that position, irradiate the nearby groundarea 120 at the user's feet and display the warning 147. The displayarea of the warning 147 is an example of a notification area between theuser and a moving body.

Note that the robot 110 may also display the warning 147 on a display ofthe audio/video output device 110 f included in the robot 110.

FIG. 35 is a flowchart illustrating an example of a process by the robot110 in the third example of the security mode “walk”.

Steps S1001 and S1007 are the same as steps S801 and S807 in FIG. 29 .

(Step S1002)

The robot 110 determines whether a person satisfying the alert conditionexists. If a person satisfying the alert condition exists (step S1002,YES), the process proceeds to step S1004, whereas if a person satisfyingthe alert condition does not exist (step S1002, NO), the processproceeds to step S1003.

(Step S1003)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 1 or lower.

The robot 110 determines the guard position 143 suitable forilluminating the nearby ground area 120. The guard position 143 in thiscase is basically the home position 134. However, the guard position 143may also be the sub-home position if it is difficult to irradiate thenearby ground area 120 from the home position 134.

(Step S1004)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 2.

The robot 110 sets the guard position 143 between the user position 130and the person position 140. First, the robot 110 sets the guard area145 between the user position 130 and the person position 140. The guardarea 145 is an area between the user position 130 and the personposition 140, and may also be an area from which the illumination device110 g can irradiate the nearby ground area 120, the notification area,or both areas with light. For example, the guard area 145 is calculatedon the basis of the user position 130, the movement direction D1, theperson position 140, and the irradiation range of the illuminationdevice 110 g. The guard area 145 may also be determined using acalculation model that accepts the user position 130, the movementdirection D1, the person position 140, and the irradiation range of theillumination device 110 g as input and outputs the guard area 145.

Alternatively, the guard area 145 may be an arc-shaped area which iscentered on the midpoint between the user position 130 and the personposition 140, and which has a thickness of prescribed size with a radiusequal to or less than the distance between the user position 130 and theperson position 140. Additionally, if the home position 134 or thesub-home position is not inside the determined guard area 145, the robot110 calculates the position inside the guard area 145 that is closest tothe robot position 131 as the guard position 143. The guard position 143is not limited to the above and may be any position inside the guardarea 145.

The state in which the home position 134 and the sub-home position arenot inside the guard area 145 corresponds to an example in which theemission position (guard position 143) is different from the currentrelative position of the robot 110. If the home position 134 or thesub-home position is inside the guard area 145, the home position 134 orthe sub-home position is treated as the guard position 143.

(Step S1005)

The robot 110 controls the moving element 110 e to move to the guardposition 143.

(Step S1006)

The robot 110 escorts the user from the guard position 143. At the sametime, the robot 110 uses the audio/video output device 110 f or theillumination device 110 g to notify the person approaching the user withan alert or a warning.

The flowchart in FIG. 35 may also include a process for irradiating thenearby ground area 120 only if the “illuminate nearby ground” functiondescribed with reference to FIG. 23 is set to “on”. The flowchart inFIG. 35 may also include a process for irradiating the danger area 122only if the “alert surroundings” function described with reference toFIG. 23 is set to “on”.

FIG. 36 is a diagram illustrating a situation in which the robot 110avoids a person approaching the user.

In FIG. 36 , the computational device 110 b of the robot 110 hascalculated the home position 134 on the basis of sensing resultsregarding the user position 130 and the movement direction D1, and iscontrolling the moving element 110 e to escort the user from the homeposition 134. The robot 110 senses the surroundings and detects a personapproaching the user from diagonally behind to the left.

The robot 110 continuously updates a predicted path 150 of the personapproaching the user according to the sensing results, and determineswhether the person will pass by the user within a prescribed time. If itis determined that the passing by will occur within the prescribed time,the robot 110 sets an avoidance position 154 inside an avoidance area152 ahead, or inside an avoidance area 151 behind, in the movementdirection D1 of the user. The avoidance position 154 may be determinedas a target position that the robot 110 attempting to change itsposition relative to the user should reach at a prescribed time.Thereafter, the robot 110 leaves the home position 134 and moves to theavoidance position 154. With this arrangement, the robot 110 can move toa position that does not obstruct the passing of the person close to theuser.

The avoidance areas 151 and 152 are areas at least a prescribed distanceaway from the predicted path 150 of the person, the predicted path 150being predicted from the person position 140 and the movement directionD2 of the person, for example. In particular, in the case where thewidth of the walkway is narrow, the avoidance area 151 is positioneddirectly behind the user, and the avoidance area 152 is positioneddirectly in front of the user.

Such behavior of the robot 110 is effective for allowing the user and aperson approaching from in front or behind the user in the movementdirection D1 of the user to pass by each other without obstructing thepath of either the user or the person. By lining up the user and therobot 110 in single-file in the movement direction D1 at the timing whenthe user and the person pass by each other, the width of the walkway isnot occupied as much and the influence on the path of the person passingby is minimized. This behavior is effective not only on narrow walkwaysbut also when the robot 110 escorts the user in places where there aremany people around the user.

If person approaching the user from either in front or behind exists,the robot 110 may move to the avoidance area 151 or the avoidance area152. For example, if a person approaching the user from behind exists,the robot 110 may move to the avoidance position 154 inside theavoidance area 151. With this arrangement, the person can be made awarethat the robot 110 is escorting the user. Furthermore, in this case, itcan be expected that the person will be discouraged from acts that wouldharm the user.

If the robot 110 does not take avoidance action, the person passing bythe user will have to change their travel route to avoid the robot 110.This change inconveniences the person and has an adverse influence onthe psychological and physical well-being of the person. Accordingly, bycausing the robot 110 to take the avoidance action described, suchadverse influence can be minimized.

The robot 110 may determine whether to set the avoidance position 154 toa position inside the avoidance area 151 or to a position inside theavoidance area 152 by considering one or more factors such as whetherthe user's risk of falling can be avoided by irradiating the nearbyground area 120 and whether the user's safety can be increased by therobot 110 positioning itself between an approacher and the user.

FIG. 37 is a flowchart illustrating an example of a process by the robot110 when the robot 110 avoids a person approaching the user. In FIG. 37, steps S1101 and S1106 are the same as steps S801 and S807 in FIG. 29 .

(Step S1102)

The robot 110 determines whether the person will pass by the robot 110within a prescribed time, and whether the distance between the personand the robot 110 is within a prescribed distance. For example, therobot 110 calculates a passing position where the person and the robot110 will pass by each other from the trail of the predicted path of theperson calculated from sensing results. The passing position is theposition on the trail where the person will come closest to the userposition 130. The robot 110 calculates the passing time by dividing thedistance on the trail from the person position 140 to the passingposition location by the speed of the person calculated from sensingresults. Additionally, if the passing time is within a prescribed time,the robot 110 may determine that the person and the robot 110 will passby each other within the prescribed time. Furthermore, if the distancebetween the person position 140 and the robot position 131 is within theprescribed distance, the robot 110 may determine that the distancebetween the user and the robot 110 is within the prescribed distance.

If the determination is positive (step S1102, YES), the process proceedsto step S1104, whereas if the determination is negative (step S1102,NO), the process proceeds to step S1103. Note that the robot 110 mayalso execute the process in step S1102 by considering only the distancebetween the person and the user, without considering the passing time.

(Step S1103)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 1 or higher.

The robot 110 sets the escort position. The escort position is basicallythe home position 134. However, the sub-home position is set as theescort position if it is difficult to irradiate the nearby ground area120 from the home position 134.

(Step S1104)

The computational device 110 b of the robot 110 sets the necessarysecurity level to 2 or higher.

The robot 110 sets the avoidance position 154. For example, if a personapproaches the user from behind, the robot 110 may set the avoidancearea 151 directly behind in the movement direction D1 of the user, andset the avoidance position 154 inside the avoidance area 151. Theavoidance area 151 is an area of prescribed size a prescribed distanceaway from the user position 130 in the opposite direction of themovement direction D1 of the user. The avoidance area 152 is an area ofprescribed size a prescribed distance away from the user position 130 inthe forward direction of the user. The avoidance position 154 is theposition inside the avoidance area 151 or the avoidance area 152 that isclosest to the escort position. However this is merely an example, andthe avoidance position 154 may be any position inside the avoidance area151 or the avoidance area 152.

(Step S1105)

The robot 110 controls the moving element 110 e to move to the escortposition or the avoidance position 154. In this way, in the case wherethe user and another person pass by each other, the robot 110 canappropriately change the position of escorting so as not to obstruct theprogress of the other person.

Next, the basic functionality of the escorting robot 110 will bedescribed. In the basic functionality, the robot 110 acquires settinginformation from the smartphone 100 (one example of a communicationterminal carried by the user) over a network. The setting information isinformation inputted by the user operating a graphical user interfacedisplayed on the settings screen 2200 or the settings screen 2300. Onthe basis of the setting information, the robot 110 sets the homeposition 134 of the robot 110, detects the user position 130 and themovement direction D1 via the sensors 110 c, and escorts the user fromthe home position 134. The home position 134 is a standard relativeposition where the robot is to be positioned by default.

FIG. 38 is a diagram illustrating an example of the basic functionalityof the escorting robot 110. In this example, the robot 110 is positionedat the robot position 131 on the left side of the user position 130. Therobot 110 uses the illumination device 110 g to display walk information160 on the road surface in front of the user position 130. The walkinformation 160 may also a goal line and a message related to the goalline. This information indicates the number of steps since the userstarted the walk or the number of steps that the user has taken today,for example. Here, it is assumed that “2000 steps”, which indicates thatthe user has taken 2000 steps since starting the walk, is displayed asthe message related to the goal line. With this arrangement, the usercan intuitively understand that by walking to the goal line, he or shewill achieve 2000 steps. In this scenario, the computational device 110b of the robot 110 sets the necessary security level to 0.

By notifying the user with the walk information 160 that quantitativelyindicates an exercise milestone, such as a target number of steps or anachieved number of steps, the user's motivation with respect to walking(exercise) is increased. The walk information 160 is not limited to theabove, and a target number of steps for the walk, a target calorieexpenditure for the walk, a target walking time for the walk, the numberof steps taken since beginning the walk, the number of steps takentoday, the remaining number of steps until today's target number ofsteps is reached, the user's heart rate, the user's calorie expenditure,the elapsed time of the walk, and/or the predicted time remaining on thewalk may also be displayed.

The robot 110 may also use the audio/video output device 110 f to outputaudio information or video information indicating the target number ofsteps, the remaining number of steps until the target number of steps isreached, the calorie expenditure, and the elapsed time. Furthermore, therobot 110 may use the audio/video output device 110 f to output speechexpressing a message of encouragement to the user, such as “Keepgoing!”.

(Modifications)

(1) The robot 110 may also have wheels instead of the legs 17. In thiscase, the robot 110 may drive the wheels to escort the user.Furthermore, the robot 110 may also have the legs 17 and wheels.

(2) The actions of the user escorted by the robot 110 are not limited towalking and may also be movement involving the user's legs, such asjogging, or movement involving the user's hands, such as movement bywheelchair or canoe, but does not include movement by car or train inwhich the user's amount of exercise is excessively low.

(3) FIG. 36 illustrates a person as an example of a moving body thatapproaches the user, but the moving body is not limited to a person andmay also be a car, a bicycle, or a flight vehicle such as a drone.

(4) In the description of FIG. 6 and thereafter, two modes (at home andwalk) are described as the security modes activated on the basis of thelocation of the user and the robot 110, but the security modes are notlimited thereto, and there may also be three or more modes (for example,at home, walk, and general going out). Also, the security mode “at home”applied when the user is at home in the above description may be a modein which security is not provided.

The present disclosure can attain a fusion between cyberspace andphysical space, and therefore is useful as a fundamental technology fora wide variety of industrial applications.

What is claimed is:
 1. A method for controlling a robot in a robotcontrol system including a server that communicates with the robot of auser and stores map information indicating a home of the user, themethod comprising: acquiring, from an external computer, requestinformation indicating a request for the robot to keep within aprescribed distance relative to a location of the user of the robot andto collect surrounding information about the user when the user isoutside a prescribed area including the home of the user; acquiring,from the robot, location information indicating a location of the robotand confirmation information indicating that the robot is near the user;and transmitting, to the robot, a command for changing a setting of therobot from a first specification to a second specification in responseto the request in a case where it is determined, on a basis of the mapinformation, the location information, and the confirmation information,that the user of the robot is accompanying the robot outside theprescribed area including the home of the user, wherein the firstspecification enables the robot to collect the surrounding informationabout the user inside the prescribed area including the home of theuser, and the second specification enables the robot to collect thesurrounding information about the user outside the prescribed areaincluding the home of the user.
 2. The method according to claim 1,wherein the prescribed area including the home includes an areaindicating a location and planar extent of the home.
 3. The methodaccording to claim 1, wherein the prescribed area including the homeincludes an area within a prescribed distance from the home.
 4. Themethod according to claim 1, wherein the prescribed area including thehome includes an area which includes the home and which is set by theuser.
 5. The method according to claim 1, wherein the prescribed areaincluding the home includes an area which includes the home and which isset in the robot control system.
 6. The method according to claim 1,wherein the confirmation information indicates that the robot hasdetermined that the robot is near the user on a basis of a result ofrecognizing the user by image recognition.
 7. The method according toclaim 1, wherein the confirmation information indicates that the robothas determined that the robot is near the user on a basis of a result ofacquiring, from a terminal carried by the user, current locationinformation about the terminal.
 8. The method according to claim 1,wherein the confirmation information includes second locationinformation which indicates a location of the user from the robot andwhich is determined by the robot on a basis of a result of recognizingthe user.
 9. The method according to claim 1, wherein the changedsetting of the robot includes a setting enabling a mobility skillprovided in the robot to move at a higher speed compared to a case wherethe robot is inside the home of the user.
 10. The method according toclaim 1, wherein the changed setting of the robot includes a settingthat turns on illumination equipment provided in the robot to illuminatethe ground near the feet of the user who is moving outside the home. 11.The method according to claim 1, wherein the command includes aninstruction causing the robot to output to the robot control system,after the setting of the robot is changed, an indication that thesetting of the robot has been changed.
 12. The method according to claim1, wherein in a case where it is confirmed that the external computerhas access rights for making the robot perform the request indicated bythe request information, the command is transmitted to the robot. 13.The method according to claim 12, wherein an indication of whether theexternal computer has the access rights is registered in the server, andthe external computer is confirmed to have the access rights on a basisof the registration.
 14. The method according to claim 1, wherein themap information indicating the home of the user includes latitude andlongitude information about the home of the user.
 15. The methodaccording to claim 1, wherein the surrounding information about the usercollected by the robot is acquired from the robot.
 16. The methodaccording to claim 14, wherein the surrounding information about theuser is transmitted to the external computer.
 17. A method forcontrolling a robot in a robot control system including a server thatcommunicates with the robot of a user, the method comprising: acquiring,from an external computer, request information indicating a request forthe robot to keep within a prescribed distance relative to a location ofthe user of the robot and to collect surrounding information about theuser when the user is outside a prescribed area including a home of theuser; acquiring, from the robot, event information indicating that theuser of the robot has moved outside the prescribed area including thehome of the user while accompanying the robot, the event informationbeing generated by the robot on a basis of i) map information that isstored in the robot, ii) location information and iii) confirmationinformation, wherein i) the map information indicates the home of theuser, ii) the location information indicates the location of the robot,and iii) the confirmation information indicates that the robot is nearthe user on a basis of detection by the robot; and transmitting, to therobot on a basis of the event information, a command for changing asetting of the robot from a first specification to a secondspecification in response to the request, wherein the firstspecification enables the robot to collect the surrounding informationabout the user inside the prescribed area including the home of theuser, and the second specification enables the robot to collect thesurrounding information about the user moving outside the prescribedarea including the home of the user.
 18. A method for controlling arobot in a robot control system including a server that communicateswith a robot of a user and stores map information indicating a home ofthe user, the method comprising: acquiring, from the robot, locationinformation indicating a location of the robot and confirmationinformation indicating that the robot is near the user; outputting themap information, the location information, and the confirmationinformation to an external computer; acquiring, from the externalcomputer, request information indicating a request for the robot to keepwithin a prescribed distance relative to a location of the user and tocollect surrounding information about the user when the user of therobot is outside a prescribed area including the home of the user, therequest information being acquired from the external computer in a casewhere it is determined at the external computer, on a basis of the mapinformation, the location information, and the confirmation information,that the user of the robot has moved outside the prescribed areaincluding the home of the user while accompanying the robot; andtransmitting, to the robot on a basis of the request, a command forchanging a setting of the robot from a first specification to a secondspecification, the first specification enabling the robot to collect thesurrounding information about the user inside the prescribed areaincluding the home of the user, and the second specification enablingthe robot to collect the surrounding information about the user movingoutside the prescribed area including the home of the user.
 19. Themethod according to claim 18, wherein the prescribed area including thehome includes an area indicating a location and planar extent of thehome.
 20. The method according to claim 18, wherein the prescribed areaincluding the home includes an area within a prescribed distance fromthe home.
 21. The method according to claim 18, wherein the prescribedarea including the home includes an area which includes the home andwhich is set by the user.
 22. The method according to claim 18, whereinthe prescribed area including the home includes an area which includesthe home and which is set in the robot control system.
 23. The methodaccording to claim 18, wherein the confirmation information indicatesthat the robot has determined that the robot is near the user on a basisof a result of recognizing the user by image recognition.
 24. The methodaccording to claim 18, wherein the confirmation information indicatesthat the robot has determined that the robot is near the user on a basisof a result of acquiring, from a terminal carried by the user, currentlocation information about the terminal.
 25. The method according toclaim 18, wherein the confirmation information includes second locationinformation which indicates a location of the user from the robot andwhich is determined by the robot on a basis of a result of recognizingthe user.
 26. The method according to claim 18, wherein the changedsetting of the robot includes a setting enabling a mobility skillprovided in the robot to move at a higher speed compared to a case wherethe robot is inside the home of the user.
 27. The method according toclaim 18, wherein the changed setting of the robot includes a settingthat turns on illumination equipment provided in the robot to illuminatethe ground near the feet of the user who is moving outside the home ofthe user.
 28. The method according to claim 18, wherein the commandincludes an instruction causing the robot to output to the robot controlsystem, after the setting of the robot is changed, an indication thatthe setting of the robot has been changed.
 29. The method according toclaim 18, wherein in a case where it is confirmed that the externalcomputer has access rights for making the robot perform the requestindicated by the request information, the command is transmitted to therobot.
 30. The method according to claim 29, wherein an indication ofwhether the external computer has the access rights is registered in theserver, and the external computer is confirmed to have the access rightson a basis of the registration.
 31. The method according to claim 18,wherein the map information indicating the home of the user includeslatitude and longitude information about the home of the user.
 32. Themethod according to claim 18, wherein the surrounding information aboutthe user collected by the robot is acquired from the robot.
 33. Themethod according to claim 32, wherein the surrounding information aboutthe user is transmitted to the external computer.
 34. A method forcontrolling a robot in a robot control system including a server thatcommunicates with a robot of a user, the method comprising: acquiring,from the robot, event information indicating that the user of the robothas moved outside a prescribed area including the home of the user whileaccompanying the robot, the event information being generated by therobot on a basis of i) map information that is stored in the robot, ii)location information and iii) confirmation information, wherein i) themap information indicates the home of the user, ii) the locationinformation indicates the location of the robot, and iii) theconfirmation information indicates that the robot is near the user on abasis of detection by the robot; outputting the event information to theexternal computer; acquiring, from the external computer in response tothe event information, request information indicating a request for therobot to keep within a prescribed distance relative to a location of theuser and to collect surrounding information about the user when the userof the robot is outside the prescribed area including the home of theuser; and transmitting, to the robot on a basis of the request, acommand for changing a setting of the robot from a first specificationto a second specification, the first specification enabling the robot tocollect the surrounding information about the user inside the prescribedarea including the home of the user, and the second specificationenabling the robot to collect the surrounding information about the usermoving outside the prescribed area including the home of the user.
 35. Amethod for controlling a robot that communicates with an externalcomputer, using map information indicating a home of a user, the methodcomprising: acquiring, from the external computer, request informationindicating a request for the robot to keep within a prescribed distancerelative to a location of the user of the robot and to collectsurrounding information about the user when the user is outside aprescribed area including the home of the user; acquiring, on a basis ofat least one internal sensor of the robot, location informationindicating a location of the robot and confirmation informationindicating that the robot is near the user; and changing a setting ofthe robot from a first specification to a second specification inresponse to the request in a case where it is determined, on a basis ofthe map information, the location information, and the confirmationinformation, that the user of the robot is accompanying the robotoutside the prescribed area including the home of the user, wherein thefirst specification enables the robot to collect the surroundinginformation about the user inside the prescribed area including the homeof the user, and the second specification enables the robot to collectthe surrounding information about the user outside the prescribed areaincluding the home of the user.
 36. A robot comprising: a main body; atleast one pair of legs or wheels; the at least one internal sensor; aprocessor; and a memory storing a program causing the processor toexecute the method according to claim
 35. 37. The method according toclaim 35, wherein the external computer is a communication terminalcarried by the user.
 38. The method according to claim 35, wherein theexternal computer is a third-party server.
 39. A non-transitorycomputer-readable medium storing a program causing a computer in therobot to execute the method according to claim 35.